Nov 02 11:53:39.237530 ip-10-0-135-24 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Nov 02 11:53:39.237544 ip-10-0-135-24 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Nov 02 11:53:39.237553 ip-10-0-135-24 systemd[1]: kubelet.service: Failed with result 'resources'.
Nov 02 11:53:39.237929 ip-10-0-135-24 systemd[1]: Failed to start Kubernetes Kubelet.
Nov 02 11:53:49.486839 ip-10-0-135-24 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Nov 02 11:53:49.486853 ip-10-0-135-24 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8f3faf22f25f47cba978245137e74cd0 --
Nov 02 11:55:30.995609 ip-10-0-135-24 systemd[1]: Starting Kubernetes Kubelet...
Nov 02 11:55:31.412149 ip-10-0-135-24 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 11:55:31.412149 ip-10-0-135-24 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 02 11:55:31.412149 ip-10-0-135-24 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 11:55:31.412149 ip-10-0-135-24 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 11:55:31.420143 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420054    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 11:55:31.420143 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420137    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 11:55:31.420143 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420141    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 11:55:31.420143 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420146    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420152    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420155    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420159    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420162    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420165    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420167    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420170    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420173    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420175    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420178    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420181    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420184    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420186    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420189    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420192    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420194    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420197    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420200    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420202    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 11:55:31.420337 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420205    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420207    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420210    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420214    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420216    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420224    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420227    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420230    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420232    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420235    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420238    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420240    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420243    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420246    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420249    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420252    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420255    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420257    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420260    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420263    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 11:55:31.420803 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420266    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420268    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420271    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420274    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420276    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420279    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420282    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420284    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420287    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420290    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420293    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420314    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420318    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420322    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420330    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420333    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420336    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420339    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420344    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 11:55:31.421321 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420348    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420352    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420355    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420358    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420361    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420364    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420369    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420372    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420375    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420378    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420380    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420383    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420386    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420389    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420391    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420394    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420397    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420400    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420402    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420405    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 11:55:31.421769 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420408    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420412    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420415    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420418    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420421    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420424    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420427    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420430    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420432    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420435    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420438    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420440    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420443    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420445    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420448    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420450    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420453    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420456    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420459    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420462    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 11:55:31.422252 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420464    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420468    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420470    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.420473    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421263    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421270    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421273    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421278    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421282    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421285    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421288    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421290    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421293    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421310    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421314    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421317    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421319    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421322    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421325    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 11:55:31.422768 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421328    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421330    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421333    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421336    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421339    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421342    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421344    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421348    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421350    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421353    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421356    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421359    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421362    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421364    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421367    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421370    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421373    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421376    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421378    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421381    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 11:55:31.423229 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421384    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421386    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421389    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421392    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421394    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421397    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421399    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421402    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421405    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421408    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421411    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421413    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421416    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421418    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421421    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421423    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421427    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421432    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421435    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 11:55:31.423719 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421439    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421441    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421444    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421447    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421450    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421454    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421456    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421459    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421462    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421464    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421467    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421469    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421472    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421474    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421477    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421480    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421482    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421485    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421487    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 11:55:31.424190 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421491    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421493    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421496    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421499    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421502    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421504    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421507    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421510    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421512    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421515    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421518    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421520    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421523    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421526    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421529    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421531    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421534    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421537    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421541    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421544    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 11:55:31.424662 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421546    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421549    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421552    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421554    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421557    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421559    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421562    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421564    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421567    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421569    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421572    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421574    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.421577    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422078    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422084    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422087    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422090    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422093    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422096    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 11:55:31.425127 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422099    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422102    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422104    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422107    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422110    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422112    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422116    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422118    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422121    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422124    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422127    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422129    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422132    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422135    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422137    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422140    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422143    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422153    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422156    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422159    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 11:55:31.425586 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422161    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 11:55:31.426068 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422164    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 11:55:31.426068 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422166    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 11:55:31.426068 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422169    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 11:55:31.426068 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422172    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 11:55:31.426068 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422175    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 11:55:31.426068 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422179    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 11:55:31.426068 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422182    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 11:55:31.426068 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422185    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 11:55:31.426068 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422189    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 11:55:31.426068 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422192    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 11:55:31.426068 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422195    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 11:55:31.426068 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422199    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 11:55:31.426068 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422201    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 11:55:31.426068 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422204    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 11:55:31.426068 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422207    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 11:55:31.426068 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422210    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 11:55:31.426068 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422212    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 11:55:31.426068 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422215    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422218    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422221    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422224    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422227    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422230    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422232    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422235    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422238    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422240    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422243    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422246    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422248    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422251    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422253    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422256    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422259    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422262    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422264    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422267    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 11:55:31.426509 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422270    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422272    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422274    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422277    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422280    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422282    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422285    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422288    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422290    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422307    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422310    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422312    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422315    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422318    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422321    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422324    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422327    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422330    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422332    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422335    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 11:55:31.426973 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422338    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422341    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422343    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422346    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422348    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422351    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422354    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422357    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422360    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422363    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422365    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422368    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422370    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422373    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422375    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422378    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422380    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422389    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422392    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 11:55:31.427475 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422394    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422397    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.422399    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423418    2580 flags.go:64] FLAG: --address="0.0.0.0"
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423427    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423439    2580 flags.go:64] FLAG: --anonymous-auth="true"
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423443    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423448    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423451    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423455    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423459    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423463    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423470    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423474    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423478    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423481    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423484    2580 flags.go:64] FLAG: --cgroup-root=""
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423487    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423490    2580 flags.go:64] FLAG: --client-ca-file=""
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423493    2580 flags.go:64] FLAG: --cloud-provider="external"
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423496    2580 flags.go:64] FLAG: --cluster-dns="[]"
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423503    2580 flags.go:64] FLAG: --cluster-domain=""
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423506    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423509    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 02 11:55:31.427916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423512    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423515    2580 flags.go:64] FLAG: --container-log-max-files="5"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423523    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423526    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423529    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423532    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423535    2580 flags.go:64] FLAG: --contention-profiling="false"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423544    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423547    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423550    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423553    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423558    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423561    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423567    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423570    2580 flags.go:64] FLAG: --enable-load-reader="false"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423575    2580 flags.go:64] FLAG: --enable-server="true"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423578    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423585    2580 flags.go:64] FLAG: --event-burst="100"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423589    2580 flags.go:64] FLAG: --event-qps="50"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423591    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423594    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423597    2580 flags.go:64] FLAG: --eviction-hard=""
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423602    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423605    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423608    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 02 11:55:31.428491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423611    2580 flags.go:64] FLAG: --eviction-soft=""
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423614    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423617    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423620    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423623    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423626    2580 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423629    2580 flags.go:64] FLAG: --fail-swap-on="true"
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423631    2580 flags.go:64] FLAG: --feature-gates=""
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423636    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423639    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423642    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423645    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423648    2580 flags.go:64] FLAG: --healthz-port="10248"
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423651    2580 flags.go:64] FLAG: --help="false"
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423654    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423667    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423671    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423674    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423677    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423681    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423685    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423688    2580 flags.go:64] FLAG: --image-service-endpoint=""
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423692    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 02 11:55:31.429130 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423695    2580 flags.go:64] FLAG: --kube-api-burst="100"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423698    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423702    2580 flags.go:64] FLAG: --kube-api-qps="50"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423704    2580 flags.go:64] FLAG: --kube-reserved=""
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423707    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423710    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423713    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423716    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423719    2580 flags.go:64] FLAG: --lock-file=""
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423722    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423725    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423728    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423734    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423737    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423740    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423743    2580 flags.go:64] FLAG: --logging-format="text"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423746    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423750    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423752    2580 flags.go:64] FLAG: --manifest-url=""
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423755    2580 flags.go:64] FLAG: --manifest-url-header=""
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423760    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423763    2580 flags.go:64] FLAG: --max-open-files="1000000"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423767    2580 flags.go:64] FLAG: --max-pods="110"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423770    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423773    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 02 11:55:31.429728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423781    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423785    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423788    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423791    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423795    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423802    2580 flags.go:64] FLAG: --node-status-max-images="50"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423807    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423810    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423813    2580 flags.go:64] FLAG: --pod-cidr=""
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423816    2580 flags.go:64] FLAG: --pod-manifest-path=""
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423819    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423822    2580 flags.go:64] FLAG: --pods-per-core="0"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423825    2580 flags.go:64] FLAG: --port="10250"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423828    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423831    2580 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-03b0be0e6fd76ab82"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423834    2580 flags.go:64] FLAG: --qos-reserved=""
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423837    2580 flags.go:64] FLAG: --read-only-port="10255"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423840    2580 flags.go:64] FLAG: --register-node="true"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423843    2580 flags.go:64] FLAG: --register-with-taints=""
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423847    2580 flags.go:64] FLAG: --registry-burst="10"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423850    2580 flags.go:64] FLAG: --registry-qps="5"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423852    2580 flags.go:64] FLAG: --reserved-cpus=""
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423855    2580 flags.go:64] FLAG: --reserved-memory=""
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423859    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423863    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423866    2580 flags.go:64] FLAG: --rotate-certificates="false"
Nov 02 11:55:31.430386 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423868    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423871    2580 flags.go:64] FLAG: --runonce="false"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423874    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423878    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423881    2580 flags.go:64] FLAG: --seccomp-default="false"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423883    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423886    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423895    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423898    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423902    2580 flags.go:64] FLAG: --storage-driver-password="root"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423906    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423909    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423913    2580 flags.go:64] FLAG: --storage-driver-user="root"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423916    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423919    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423922    2580 flags.go:64] FLAG: --system-cgroups=""
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423925    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423930    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423933    2580 flags.go:64] FLAG: --tls-cert-file=""
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423936    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423943    2580 flags.go:64] FLAG: --tls-min-version=""
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423945    2580 flags.go:64] FLAG: --tls-private-key-file=""
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423948    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423951    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423954    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 02 11:55:31.430987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423957    2580 flags.go:64] FLAG: --v="2"
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423961    2580 flags.go:64] FLAG: --version="false"
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423965    2580 flags.go:64] FLAG: --vmodule=""
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423969    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.423973    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424106    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424111    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424115    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424118    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424122    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424124    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424127    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424130    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424133    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424135    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424138    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424141    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424145    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424148    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424152    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 11:55:31.431588 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424155    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424157    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424161    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424163    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424166    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424169    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424171    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424174    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424177    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424180    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424183    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424185    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424188    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424191    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424193    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424197    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424199    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424202    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424205    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 11:55:31.432132 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424207    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424210    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424212    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424215    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424218    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424220    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424223    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424225    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424228    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424231    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424235    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424238    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424242    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424244    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424247    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424250    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424253    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424256    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424258    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424261    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 11:55:31.432599 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424263    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424266    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424269    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424271    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424274    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424276    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424279    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424282    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424285    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424287    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424290    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424292    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424309    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424312    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424315    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424317    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424320    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424323    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424325    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424328    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 11:55:31.433109 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424331    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424333    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424337    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424340    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424344    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424347    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424350    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424352    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424355    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424358    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424361    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424363    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424366    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424368    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424371    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424374    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424377    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424379    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424382    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 11:55:31.433585 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424385    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 11:55:31.434042 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424387    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 11:55:31.434042 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424390    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 11:55:31.434042 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424393    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 11:55:31.434042 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424395    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 11:55:31.434042 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424399    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 11:55:31.434042 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424403    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 11:55:31.434042 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424406    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 11:55:31.434042 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424409    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 11:55:31.434042 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424412    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 11:55:31.434042 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424414    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 11:55:31.434042 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424417    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 11:55:31.434042 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424420    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 11:55:31.434042 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.424446    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 11:55:31.434042 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424485    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 11:55:31.434042 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424490    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424496    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424502    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424506    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424509    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424512    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424514    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424517    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424520    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424522    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424525    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424528    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424531    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424533    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424536    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424539    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424542    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424544    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424547    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 11:55:31.434420 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424549    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424552    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424555    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424558    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424560    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424563    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424565    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424568    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424571    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424573    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424576    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424579    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424581    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424584    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424588    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424592    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424595    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424598    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424600    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424603    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 11:55:31.434875 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424605    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424608    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424611    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424613    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424616    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424619    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424622    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424625    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424627    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424630    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424632    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424635    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424638    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424640    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424643    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424646    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424648    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424651    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424653    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424656    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 11:55:31.435355 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424659    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424661    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424664    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424667    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424670    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424672    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424676    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424680    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424683    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424685    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424688    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424691    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424693    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424696    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424698    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424701    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424703    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424706    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424708    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424711    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 11:55:31.435813 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424713    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424716    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424718    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424721    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424723    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424726    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424728    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424731    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424733    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424736    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424739    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424742    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424744    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424747    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424749    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424752    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424754    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424757    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424759    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 11:55:31.436268 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424763    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424767    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424770    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424773    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424777    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424779    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.424782    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.431917    2580 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.431933    2580 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.431950    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.431954    2580 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432017    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432024    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432028    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432031    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432034    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432037    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432040    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432043    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 11:55:31.436725 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432046    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432049    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432052    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432055    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432057    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432060    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432063    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432066    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432068    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432071    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432074    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432076    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432079    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432082    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432084    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432087    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432090    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432092    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432095    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432098    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 11:55:31.437171 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432101    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432110    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432113    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432116    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432118    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432121    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432124    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432127    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432129    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432132    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432135    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432138    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432141    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432144    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432147    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432150    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432152    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432155    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432157    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 11:55:31.437720 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432160    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432163    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432166    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432168    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432171    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432174    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432176    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432179    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432182    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432185    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432187    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432190    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432192    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432195    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432198    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432208    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432211    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432214    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432216    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432219    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 11:55:31.438197 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432222    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432224    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432227    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432229    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432232    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432235    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432237    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432240    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432243    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432245    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432247    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432250    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432253    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432255    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432258    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432260    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432263    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432266    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432269    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432272    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 11:55:31.438680 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432274    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 11:55:31.439144 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432277    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 11:55:31.439144 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432281    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 11:55:31.439144 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432286    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 11:55:31.439144 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432290    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 11:55:31.439144 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432293    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 11:55:31.439144 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432309    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 11:55:31.439144 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432313    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 11:55:31.439144 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432322    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 11:55:31.439144 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432325    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 11:55:31.439144 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432328    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 11:55:31.439144 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432331    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 11:55:31.439144 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432333    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 11:55:31.439144 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432336    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 11:55:31.439144 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432340    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 11:55:31.439144 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432343    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 11:55:31.439144 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432347    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 11:55:31.439144 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432350    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 11:55:31.439144 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432354    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 11:55:31.439578 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.432381    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 11:55:31.439578 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432566    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 11:55:31.439578 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432571    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 11:55:31.439578 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432575    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 11:55:31.439578 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432579    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 11:55:31.439578 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432582    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 11:55:31.439578 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432585    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 11:55:31.439578 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432587    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 11:55:31.439578 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432590    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 11:55:31.439578 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432593    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 11:55:31.439578 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432596    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 11:55:31.439578 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432599    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 11:55:31.439578 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432602    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 11:55:31.439578 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432604    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 11:55:31.439578 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432608    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 11:55:31.439578 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432610    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432613    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432616    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432619    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432621    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432624    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432626    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432634    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432637    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432640    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432643    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432646    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432648    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432650    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432653    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432656    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432658    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432661    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432663    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432666    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 11:55:31.439953 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432669    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432671    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432674    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432677    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432679    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432682    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432684    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432687    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432690    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432692    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432695    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432698    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432702    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432705    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432709    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432712    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432715    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432718    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432721    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 11:55:31.440432 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432724    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432732    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432735    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432737    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432740    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432742    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432745    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432747    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432750    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432753    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432755    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432758    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432760    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432763    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432765    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432768    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432771    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432773    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432776    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432779    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 11:55:31.440893 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432781    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432784    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432786    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432796    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432799    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432802    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432804    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432807    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432810    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432813    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432815    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432818    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432821    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432823    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432832    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432835    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432837    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432840    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432842    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432845    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 11:55:31.441377 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432847    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 11:55:31.441833 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432850    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 11:55:31.441833 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432852    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 11:55:31.441833 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432855    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 11:55:31.441833 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432857    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 11:55:31.441833 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432860    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 11:55:31.441833 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432863    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 11:55:31.441833 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432865    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 11:55:31.441833 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432868    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 11:55:31.441833 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432870    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 11:55:31.441833 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432873    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 11:55:31.441833 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432875    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 11:55:31.441833 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432878    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 11:55:31.441833 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.432901    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 11:55:31.441833 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432937    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 11:55:31.441833 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432942    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 11:55:31.441833 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432945    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432950    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432953    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432957    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432960    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432963    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432966    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432969    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432972    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432974    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432977    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432987    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432990    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432993    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432996    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.432999    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433001    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433004    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433006    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 11:55:31.442220 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433009    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433012    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433014    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433017    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433020    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433022    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433025    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433027    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433030    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433033    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433039    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433042    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433045    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433047    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433052    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433055    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433057    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433060    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433063    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433065    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 11:55:31.442735 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433068    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433071    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433073    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433076    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433079    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433087    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433090    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433092    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433095    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433097    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433100    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433103    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433105    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433109    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433112    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433116    2580 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433119    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433121    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433124    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433127    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 11:55:31.443569 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433130    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433132    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433135    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433137    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433140    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433142    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433147    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433150    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433152    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433155    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433158    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433160    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433163    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433166    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433169    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433171    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433174    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433177    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433185    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 11:55:31.444312 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433188    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433190    2580 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433193    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433196    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433198    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433201    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433203    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433206    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433209    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433212    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433214    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433217    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433220    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433223    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433226    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433229    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433231    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433234    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433236    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 11:55:31.444792 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433242    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 11:55:31.445347 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433245    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 11:55:31.445347 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433248    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 11:55:31.445347 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433251    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 11:55:31.445347 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433254    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 11:55:31.445347 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433257    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 11:55:31.445347 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:31.433260    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 11:55:31.445347 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.433330    2580 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 02 11:55:31.445347 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.437383    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 02 11:55:31.445347 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.438390    2580 server.go:1014] "Starting client certificate rotation"
Nov 02 11:55:31.445347 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.438510    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 11:55:31.445347 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.438544    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 11:55:31.445347 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.440700    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 11:55:31.445686 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.445668    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 11:55:31.454894 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.454875    2580 log.go:25] "Validated CRI v1 runtime API"
Nov 02 11:55:31.458716 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.458700    2580 log.go:25] "Validated CRI v1 image API"
Nov 02 11:55:31.459528 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.459502    2580 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 02 11:55:31.462259 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.462241    2580 fs.go:135] Filesystem UUIDs: map[58732fc7-915a-4274-9f43-b2de0be07202:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7fdad5ae-d3e0-4035-acc0-4b531e2ebf33:/dev/nvme0n1p4]
Nov 02 11:55:31.462327 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.462259    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 02 11:55:31.467218 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.467115    2580 manager.go:217] Machine: {Timestamp:2026-11-02 11:55:31.465783531 +0000 UTC m=+0.350607803 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099793 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27576d08b86e6a3c9188c18fdd4451 SystemUUID:ec27576d-08b8-6e6a-3c91-88c18fdd4451 BootID:8f3faf22-f25f-47cb-a978-245137e74cd0 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:01:4b:30:61:b7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:01:4b:30:61:b7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:08:f7:0a:c2:7e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 02 11:55:31.467218 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.467216    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 02 11:55:31.467329 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.467290    2580 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 02 11:55:31.470806 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.470781    2580 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 02 11:55:31.470962 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.470807    2580 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-24.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 02 11:55:31.471041 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.470968    2580 topology_manager.go:143] "Creating topology manager with none policy"
Nov 02 11:55:31.471041 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.470980    2580 container_manager_linux.go:311] "Creating device plugin manager"
Nov 02 11:55:31.471041 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.470993    2580 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 11:55:31.471917 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.471900    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 11:55:31.473175 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.473163    2580 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 02 11:55:31.477717 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.477703    2580 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 02 11:55:31.477860 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.477849    2580 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 02 11:55:31.478690 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.478679    2580 kubelet.go:493] "Attempting to sync node with API server"
Nov 02 11:55:31.478758 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.478697    2580 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 02 11:55:31.478758 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.478711    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 02 11:55:31.478758 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.478724    2580 kubelet.go:405] "Adding apiserver pod source"
Nov 02 11:55:31.478758 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.478736    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 02 11:55:31.479165 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.479142    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 11:55:31.481535 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.481505    2580 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 02 11:55:31.490267 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.490246    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 11:55:31.491146 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.491134    2580 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 02 11:55:31.491192 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.491164    2580 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 02 11:55:31.491331 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.491320    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 02 11:55:31.491373 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.491336    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 02 11:55:31.491373 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.491341    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 02 11:55:31.491373 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.491347    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 02 11:55:31.491373 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.491353    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 02 11:55:31.491373 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.491358    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 02 11:55:31.492116 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.492108    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 02 11:55:31.492143 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.492117    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 02 11:55:31.492143 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.492125    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 02 11:55:31.492143 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.492131    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 02 11:55:31.492143 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.492139    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 02 11:55:31.492340 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.492152    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 02 11:55:31.493696 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.493682    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 02 11:55:31.493696 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.493693    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 02 11:55:31.497420 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.497409    2580 server.go:1263] "Started kubelet"
Nov 02 11:55:31.497752 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.497731    2580 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 02 11:55:31.497996 ip-10-0-135-24 systemd[1]: Started Kubernetes Kubelet.
Nov 02 11:55:31.498681 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.498667    2580 server.go:317] "Adding debug handlers to kubelet server"
Nov 02 11:55:31.499407 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.499351    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 02 11:55:31.499494 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.499425    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 02 11:55:31.501521 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.501447    2580 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 02 11:55:31.504930 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:31.504911    2580 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 02 11:55:31.507136 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.507121    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 02 11:55:31.508385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.508371    2580 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 02 11:55:31.508523 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:31.508488    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-24.us-west-1.compute.internal\" not found"
Nov 02 11:55:31.509155 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.509122    2580 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 02 11:55:31.509155 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.509134    2580 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 02 11:55:31.509155 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.509134    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 02 11:55:31.509352 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.509212    2580 reconstruct.go:97] "Volume reconstruction finished"
Nov 02 11:55:31.509352 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.509220    2580 reconciler.go:29] "Reconciler: start to sync state"
Nov 02 11:55:31.509505 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.509490    2580 factory.go:55] Registering systemd factory
Nov 02 11:55:31.509537 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.509508    2580 factory.go:223] Registration of the systemd container factory successfully
Nov 02 11:55:31.509702 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.509689    2580 factory.go:153] Registering CRI-O factory
Nov 02 11:55:31.509737 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.509703    2580 factory.go:223] Registration of the crio container factory successfully
Nov 02 11:55:31.509765 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.509749    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 02 11:55:31.509765 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.509762    2580 factory.go:103] Registering Raw factory
Nov 02 11:55:31.509833 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.509772    2580 manager.go:1199] Started watching for new ooms in manager
Nov 02 11:55:31.510962 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.510943    2580 manager.go:319] Starting recovery of all containers
Nov 02 11:55:31.522576 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.522560    2580 manager.go:324] Recovery completed
Nov 02 11:55:31.524290 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:31.524268    2580 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": readdirent /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Nov 02 11:55:31.527064 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.527051    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 11:55:31.528680 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.528651    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 11:55:31.528753 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.528696    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 11:55:31.528753 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.528710    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 11:55:31.529220 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.529205    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 11:55:31.529220 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.529215    2580 cpu_manager.go:226] "Starting" policy="none"
Nov 02 11:55:31.529341 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.529223    2580 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 02 11:55:31.529341 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.529239    2580 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 02 11:55:31.529928 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.529913    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 11:55:31.529991 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.529946    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 11:55:31.529991 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.529962    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 11:55:31.531458 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.531444    2580 policy_none.go:50] "Start"
Nov 02 11:55:31.531523 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.531462    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 02 11:55:31.531523 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.531474    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 02 11:55:31.533502 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.533489    2580 policy_none.go:44] "Start"
Nov 02 11:55:31.586772 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.567781    2580 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 02 11:55:31.586772 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.576609    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 02 11:55:31.586772 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.577679    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 02 11:55:31.586772 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.577696    2580 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 02 11:55:31.586772 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.577754    2580 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 02 11:55:31.586772 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:31.577801    2580 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 02 11:55:31.586772 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.582377    2580 manager.go:341] "Starting Device Plugin manager"
Nov 02 11:55:31.586772 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:31.583042    2580 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 02 11:55:31.586772 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.583054    2580 server.go:87] "Starting device plugin registration server"
Nov 02 11:55:31.586772 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.583317    2580 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 02 11:55:31.586772 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.583334    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 02 11:55:31.586772 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.583490    2580 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 02 11:55:31.586772 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.583554    2580 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 02 11:55:31.586772 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.583561    2580 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 02 11:55:31.586772 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:31.583955    2580 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 02 11:55:31.586772 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:31.584004    2580 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-24.us-west-1.compute.internal\" not found"
Nov 02 11:55:31.641972 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.641883    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-24.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 02 11:55:31.641972 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:31.641883    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-24.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 02 11:55:31.678772 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.678716    2580 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal"]
Nov 02 11:55:31.678863 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.678773    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 11:55:31.680345 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.680331    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 11:55:31.680434 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.680359    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 11:55:31.680434 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.680375    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 11:55:31.683712 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.683698    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 11:55:31.683775 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.683718    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 11:55:31.683858 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.683845    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:31.683892 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.683874    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 11:55:31.684515 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.684500    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 11:55:31.684584 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.684525    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 11:55:31.684584 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.684542    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 11:55:31.684584 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.684564    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 11:55:31.684584 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.684575    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 11:55:31.684767 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.684588    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 11:55:31.684767 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.684594    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 11:55:31.684767 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.684603    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 11:55:31.684767 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.684608    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 11:55:31.684767 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.684571    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:31.686667 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.686651    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:31.686732 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.686674    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 11:55:31.687822 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.687808    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 11:55:31.687897 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.687830    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 11:55:31.687897 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.687839    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 11:55:31.690285 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:31.689404    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-24.us-west-1.compute.internal.18e4385ed1bfb7f7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-24.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-24.us-west-1.compute.internal,},FirstTimestamp:2026-11-02 11:55:31.497383927 +0000 UTC m=+0.382208200,LastTimestamp:2026-11-02 11:55:31.497383927 +0000 UTC m=+0.382208200,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-24.us-west-1.compute.internal,}"
Nov 02 11:55:31.722608 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.722584    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 02 11:55:31.722705 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.722647    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jcqdq"
Nov 02 11:55:31.730371 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:31.730356    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-24.us-west-1.compute.internal\" not found" node="ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:31.732222 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:31.732208    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-24.us-west-1.compute.internal\" not found" node="ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:31.743087 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:31.743062    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass"
Nov 02 11:55:31.743087 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:31.743075    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 02 11:55:31.743207 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:31.743105    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 02 11:55:31.743207 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:31.743092    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-24.us-west-1.compute.internal.18e4385ed39d4322  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-24.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-24.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-24.us-west-1.compute.internal,},FirstTimestamp:2026-11-02 11:55:31.528680226 +0000 UTC m=+0.413504502,LastTimestamp:2026-11-02 11:55:31.528680226 +0000 UTC m=+0.413504502,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-24.us-west-1.compute.internal,}"
Nov 02 11:55:31.743338 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:31.743218    2580 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-24.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 02 11:55:31.803486 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:31.803466    2580 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-135-24.us-west-1.compute.internal\" not found" node="ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:31.810268 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.810247    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a8a4e5ba356fe85cd903457b215689b7-config\") pod \"kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal\" (UID: \"a8a4e5ba356fe85cd903457b215689b7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:31.810376 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.810279    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b0871b91e1bc485b0c3cd650ef3a7622-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal\" (UID: \"b0871b91e1bc485b0c3cd650ef3a7622\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:31.810376 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.810318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0871b91e1bc485b0c3cd650ef3a7622-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal\" (UID: \"b0871b91e1bc485b0c3cd650ef3a7622\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:31.911367 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.911342    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b0871b91e1bc485b0c3cd650ef3a7622-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal\" (UID: \"b0871b91e1bc485b0c3cd650ef3a7622\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:31.911494 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.911380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b0871b91e1bc485b0c3cd650ef3a7622-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal\" (UID: \"b0871b91e1bc485b0c3cd650ef3a7622\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:31.911494 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.911444    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0871b91e1bc485b0c3cd650ef3a7622-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal\" (UID: \"b0871b91e1bc485b0c3cd650ef3a7622\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:31.911494 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.911481    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a8a4e5ba356fe85cd903457b215689b7-config\") pod \"kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal\" (UID: \"a8a4e5ba356fe85cd903457b215689b7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:31.911612 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.911515    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a8a4e5ba356fe85cd903457b215689b7-config\") pod \"kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal\" (UID: \"a8a4e5ba356fe85cd903457b215689b7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:31.911612 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:31.911518    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0871b91e1bc485b0c3cd650ef3a7622-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal\" (UID: \"b0871b91e1bc485b0c3cd650ef3a7622\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:31.945325 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:31.945265    2580 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-24.us-west-1.compute.internal\" not found" node="ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:32.003906 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:32.003887    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 11:55:32.004803 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:32.004790    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 11:55:32.004878 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:32.004819    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 11:55:32.004878 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:32.004830    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 11:55:32.004878 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:32.004859    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:32.035118 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:32.035103    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:32.038305 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:32.038281    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:32.060752 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:32.060639    2580 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:32.060809 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:32.060758    2580 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-24.us-west-1.compute.internal\": node \"ip-10-0-135-24.us-west-1.compute.internal\" not found"
Nov 02 11:55:32.173917 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:32.173890    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-24.us-west-1.compute.internal\" not found"
Nov 02 11:55:32.274662 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:32.274632    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-24.us-west-1.compute.internal\" not found"
Nov 02 11:55:32.375572 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:32.375546    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-24.us-west-1.compute.internal\" not found"
Nov 02 11:55:32.438035 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:32.438013    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 02 11:55:32.438474 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:32.438155    2580 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://aadda5a39e08548bea3cd7c71322c295-74c64663e2d0aa4a.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-135-24.us-west-1.compute.internal.18e4385ed39d997c\": read tcp 10.0.135.24:57224->77.112.245.159:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-135-24.us-west-1.compute.internal.18e4385ed39d997c  default   4768 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-24.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-24.us-west-1.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-24.us-west-1.compute.internal,},FirstTimestamp:2026-11-02 11:55:31 +0000 UTC,LastTimestamp:2026-11-02 11:55:31.684532284 +0000 UTC m=+0.569356556,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-24.us-west-1.compute.internal,}"
Nov 02 11:55:32.476306 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:32.476238    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-24.us-west-1.compute.internal\" not found"
Nov 02 11:55:32.576889 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:32.576870    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-24.us-west-1.compute.internal\" not found"
Nov 02 11:55:32.677136 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:32.677109    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-24.us-west-1.compute.internal\" not found"
Nov 02 11:55:32.723562 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:32.723544    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 11:55:32.724358 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:32.724346    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 11:55:32.724420 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:32.724373    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 11:55:32.724420 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:32.724383    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 11:55:32.728506 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:32.728450    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-01 11:50:31 +0000 UTC" deadline="2028-07-17 16:21:18.044684342 +0000 UTC"
Nov 02 11:55:32.728506 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:32.728479    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14956h25m45.316208128s"
Nov 02 11:55:32.770168 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:32.770152    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 11:55:32.777503 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:32.777486    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-24.us-west-1.compute.internal\" not found"
Nov 02 11:55:32.860100 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:32.860078    2580 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 11:55:32.878180 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:32.878160    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-24.us-west-1.compute.internal\" not found"
Nov 02 11:55:32.937591 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:32.937573    2580 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 11:55:32.978344 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:32.978325    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-24.us-west-1.compute.internal\" not found"
Nov 02 11:55:33.044967 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.044945    2580 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 11:55:33.097116 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:33.097092    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0871b91e1bc485b0c3cd650ef3a7622.slice/crio-37670025ba6d0060a48cbaf6065cef83c2fbd01e7a266b2180386d9fa1b75731 WatchSource:0}: Error finding container 37670025ba6d0060a48cbaf6065cef83c2fbd01e7a266b2180386d9fa1b75731: Status 404 returned error can't find the container with id 37670025ba6d0060a48cbaf6065cef83c2fbd01e7a266b2180386d9fa1b75731
Nov 02 11:55:33.097531 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:33.097512    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8a4e5ba356fe85cd903457b215689b7.slice/crio-fa8182ba90f40af301e6472b699b54e312b9b5db522fd5e2b3bf7db970775544 WatchSource:0}: Error finding container fa8182ba90f40af301e6472b699b54e312b9b5db522fd5e2b3bf7db970775544: Status 404 returned error can't find the container with id fa8182ba90f40af301e6472b699b54e312b9b5db522fd5e2b3bf7db970775544
Nov 02 11:55:33.103973 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.103957    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 11:55:33.108763 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.108750    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:33.164309 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.164284    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 11:55:33.165226 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.165215    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:55:33.216476 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.216463    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 11:55:33.479833 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.479755    2580 apiserver.go:52] "Watching apiserver"
Nov 02 11:55:33.507665 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.507646    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 02 11:55:33.573730 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.573708    2580 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 02 11:55:33.573982 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.573965    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal","openshift-multus/multus-additional-cni-plugins-mwslp","kube-system/global-pull-secret-syncer-vm62x","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk","openshift-image-registry/node-ca-xt7dv","openshift-network-diagnostics/network-check-target-2vw9j","openshift-ovn-kubernetes/ovnkube-node-7686t","openshift-dns/node-resolver-h24nt","openshift-multus/network-metrics-daemon-n4tth","openshift-cluster-node-tuning-operator/tuned-sv47s","openshift-multus/multus-fsq7c","openshift-network-operator/iptables-alerter-dqg86","kube-system/konnectivity-agent-797nn","kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal"]
Nov 02 11:55:33.577506 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.577492    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.579452 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.579439    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:33.579522 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:33.579486    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4tth" podUID="60dc0a20-c792-4343-aec8-7ff8f86955e9"
Nov 02 11:55:33.581545 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.581530    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-h24nt"
Nov 02 11:55:33.583565 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.583550    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.585618 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.585602    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.587600 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.587584    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.589886 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.589865    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xt7dv"
Nov 02 11:55:33.592443 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.592427    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-797nn"
Nov 02 11:55:33.594446 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.594430    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:33.594526 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:33.594487    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vw9j" podUID="cb734498-5d2a-40c1-9d1f-443e3e8c8f87"
Nov 02 11:55:33.596481 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.596466    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.598522 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.598507    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dqg86"
Nov 02 11:55:33.600520 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.600506    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:33.600602 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:33.600547    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm62x" podUID="6d8ba8e4-34a6-47eb-a33f-b67c52acd079"
Nov 02 11:55:33.602829 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.602799    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal" event={"ID":"a8a4e5ba356fe85cd903457b215689b7","Type":"ContainerStarted","Data":"fa8182ba90f40af301e6472b699b54e312b9b5db522fd5e2b3bf7db970775544"}
Nov 02 11:55:33.602913 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.602839    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal" event={"ID":"b0871b91e1bc485b0c3cd650ef3a7622","Type":"ContainerStarted","Data":"37670025ba6d0060a48cbaf6065cef83c2fbd01e7a266b2180386d9fa1b75731"}
Nov 02 11:55:33.610165 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.610145    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 02 11:55:33.618951 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.618934    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-cnibin\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.619041 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.618956    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a093470-b06b-49c5-9220-2db08ecfde86-cnibin\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.619041 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.618971    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbqbh\" (UniqueName: \"kubernetes.io/projected/0a093470-b06b-49c5-9220-2db08ecfde86-kube-api-access-lbqbh\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.619041 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.618986    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-modprobe-d\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.619041 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619001    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-systemd\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.619041 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619013    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-cni-bin\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.619041 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619026    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:33.619041 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619041    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-run-k8s-cni-cncf-io\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.619384 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619052    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-run-netns\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.619384 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619066    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6b350fc9-6bb7-43c4-9b89-c5157f42ca26-konnectivity-ca\") pod \"konnectivity-agent-797nn\" (UID: \"6b350fc9-6bb7-43c4-9b89-c5157f42ca26\") " pod="kube-system/konnectivity-agent-797nn"
Nov 02 11:55:33.619384 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-sysconfig\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.619384 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619116    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-etc-openvswitch\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.619384 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e81ecb7e-0219-4976-b881-8c87ab61db50-ovnkube-config\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.619384 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619180    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e81ecb7e-0219-4976-b881-8c87ab61db50-ovnkube-script-lib\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.619384 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619201    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-socket-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.619384 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-var-lib-kubelet\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.619384 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619242    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-multus-conf-dir\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.619384 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619262    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a093470-b06b-49c5-9220-2db08ecfde86-system-cni-dir\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.619384 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619291    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-sysctl-d\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.619384 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619345    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-tuned\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.619384 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619365    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-systemd-units\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.619384 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619383    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-var-lib-openvswitch\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.620014 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619404    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbnkn\" (UniqueName: \"kubernetes.io/projected/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-kube-api-access-tbnkn\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.620014 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619419    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a093470-b06b-49c5-9220-2db08ecfde86-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.620014 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619433    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-run-netns\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.620014 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619446    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.620014 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619465    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.620014 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619477    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d239eb0-492f-4335-857e-4edc754823e4-tmp-dir\") pod \"node-resolver-h24nt\" (UID: \"4d239eb0-492f-4335-857e-4edc754823e4\") " pod="openshift-dns/node-resolver-h24nt"
Nov 02 11:55:33.620014 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619497    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-kubelet\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.620014 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619512    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5ca3e4d6-0eac-4309-b049-e5f597728119-iptables-alerter-script\") pod \"iptables-alerter-dqg86\" (UID: \"5ca3e4d6-0eac-4309-b049-e5f597728119\") " pod="openshift-network-operator/iptables-alerter-dqg86"
Nov 02 11:55:33.620014 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619535    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-hostroot\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.620014 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619551    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-tmp\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.620014 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619573    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbnkr\" (UniqueName: \"kubernetes.io/projected/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-kube-api-access-lbnkr\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.620014 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619600    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-multus-socket-dir-parent\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.620014 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a093470-b06b-49c5-9220-2db08ecfde86-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.620014 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619641    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:33.620014 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619659    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-slash\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.620014 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619691    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-run-systemd\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.620640 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619727    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e81ecb7e-0219-4976-b881-8c87ab61db50-ovn-node-metrics-cert\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.620640 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-sys-fs\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.620640 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619769    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7hn7\" (UniqueName: \"kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7\") pod \"network-check-target-2vw9j\" (UID: \"cb734498-5d2a-40c1-9d1f-443e3e8c8f87\") " pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:33.620640 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsnt9\" (UniqueName: \"kubernetes.io/projected/60dc0a20-c792-4343-aec8-7ff8f86955e9-kube-api-access-tsnt9\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:33.620640 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619809    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-multus-cni-dir\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.620640 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619829    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc644b68-76cd-4326-8a15-2f8e3dd393d0-host\") pod \"node-ca-xt7dv\" (UID: \"fc644b68-76cd-4326-8a15-2f8e3dd393d0\") " pod="openshift-image-registry/node-ca-xt7dv"
Nov 02 11:55:33.620640 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619847    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ca3e4d6-0eac-4309-b049-e5f597728119-host-slash\") pod \"iptables-alerter-dqg86\" (UID: \"5ca3e4d6-0eac-4309-b049-e5f597728119\") " pod="openshift-network-operator/iptables-alerter-dqg86"
Nov 02 11:55:33.620640 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619865    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb29w\" (UniqueName: \"kubernetes.io/projected/68728c63-ace5-4888-a19b-2782fd2ba673-kube-api-access-hb29w\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.620640 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619904    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-node-log\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.620640 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-run-ovn-kubernetes\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.620640 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.619973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-device-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.620640 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620002    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk6fk\" (UniqueName: \"kubernetes.io/projected/4d239eb0-492f-4335-857e-4edc754823e4-kube-api-access-gk6fk\") pod \"node-resolver-h24nt\" (UID: \"4d239eb0-492f-4335-857e-4edc754823e4\") " pod="openshift-dns/node-resolver-h24nt"
Nov 02 11:55:33.620640 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-var-lib-cni-bin\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.620640 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620034    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-etc-kubernetes\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.620640 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620062    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a093470-b06b-49c5-9220-2db08ecfde86-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.620640 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620098    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-host\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.621395 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620120    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-registration-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.621395 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620135    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/68728c63-ace5-4888-a19b-2782fd2ba673-cni-binary-copy\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.621395 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620186    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a093470-b06b-49c5-9220-2db08ecfde86-cni-binary-copy\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.621395 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620215    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-var-lib-kubelet\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.621395 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620239    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fndj\" (UniqueName: \"kubernetes.io/projected/5ca3e4d6-0eac-4309-b049-e5f597728119-kube-api-access-7fndj\") pod \"iptables-alerter-dqg86\" (UID: \"5ca3e4d6-0eac-4309-b049-e5f597728119\") " pod="openshift-network-operator/iptables-alerter-dqg86"
Nov 02 11:55:33.621395 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fc644b68-76cd-4326-8a15-2f8e3dd393d0-serviceca\") pod \"node-ca-xt7dv\" (UID: \"fc644b68-76cd-4326-8a15-2f8e3dd393d0\") " pod="openshift-image-registry/node-ca-xt7dv"
Nov 02 11:55:33.621395 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620275    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6b350fc9-6bb7-43c4-9b89-c5157f42ca26-agent-certs\") pod \"konnectivity-agent-797nn\" (UID: \"6b350fc9-6bb7-43c4-9b89-c5157f42ca26\") " pod="kube-system/konnectivity-agent-797nn"
Nov 02 11:55:33.621395 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620325    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-run\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.621395 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620340    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-log-socket\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.621395 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620366    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-kubelet-config\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:33.621395 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620383    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d239eb0-492f-4335-857e-4edc754823e4-hosts-file\") pod \"node-resolver-h24nt\" (UID: \"4d239eb0-492f-4335-857e-4edc754823e4\") " pod="openshift-dns/node-resolver-h24nt"
Nov 02 11:55:33.621395 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620405    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-system-cni-dir\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.621395 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620425    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/68728c63-ace5-4888-a19b-2782fd2ba673-multus-daemon-config\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.621395 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620443    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-sysctl-conf\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.621395 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620466    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-run-openvswitch\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.621395 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-cni-netd\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.622082 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620498    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-dbus\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:33.622082 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620514    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-var-lib-cni-multus\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.622082 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620537    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-sys\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.622082 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620555    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-run-ovn\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.622082 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620570    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e81ecb7e-0219-4976-b881-8c87ab61db50-env-overrides\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.622082 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620601    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-run-multus-certs\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.622082 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620622    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvpj5\" (UniqueName: \"kubernetes.io/projected/fc644b68-76cd-4326-8a15-2f8e3dd393d0-kube-api-access-tvpj5\") pod \"node-ca-xt7dv\" (UID: \"fc644b68-76cd-4326-8a15-2f8e3dd393d0\") " pod="openshift-image-registry/node-ca-xt7dv"
Nov 02 11:55:33.622082 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620635    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-kubernetes\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.622082 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620657    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-lib-modules\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.622082 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620677    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g8ls\" (UniqueName: \"kubernetes.io/projected/e81ecb7e-0219-4976-b881-8c87ab61db50-kube-api-access-7g8ls\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.622082 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.620696    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-etc-selinux\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.625443 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.625428    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 02 11:55:33.625591 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.625559    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 02 11:55:33.625680 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.625630    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 02 11:55:33.625680 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.625646    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 02 11:55:33.625680 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.625655    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x7vnq\""
Nov 02 11:55:33.625680 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.625659    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 02 11:55:33.625838 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.625745    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 02 11:55:33.628645 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.628632    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 02 11:55:33.628997 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.628984    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 02 11:55:33.629621 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.629605    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-q864t\""
Nov 02 11:55:33.630877 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.630865    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2rrmx\""
Nov 02 11:55:33.667893 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.667875    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 02 11:55:33.667984 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.667906    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pz992"
Nov 02 11:55:33.717416 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717398    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 02 11:55:33.717416 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717412    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 02 11:55:33.717568 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717398    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 02 11:55:33.717568 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717506    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 02 11:55:33.717653 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717573    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 02 11:55:33.717653 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717604    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 02 11:55:33.717653 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717611    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hx94g\""
Nov 02 11:55:33.717653 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717616    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 02 11:55:33.717653 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717625    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 02 11:55:33.717806 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717664    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 02 11:55:33.717806 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717755    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 02 11:55:33.717806 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717761    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 02 11:55:33.717806 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717791    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gpdd6\""
Nov 02 11:55:33.717806 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717796    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 02 11:55:33.717982 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717768    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 02 11:55:33.717982 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717814    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 02 11:55:33.717982 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717865    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 02 11:55:33.717982 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717872    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-bjwgn\""
Nov 02 11:55:33.717982 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717875    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 02 11:55:33.717982 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717884    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 02 11:55:33.717982 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717888    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mvmxd\""
Nov 02 11:55:33.717982 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717909    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 02 11:55:33.717982 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717913    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-g4drg\""
Nov 02 11:55:33.717982 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717919    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 02 11:55:33.717982 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.717943    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4pvvq\""
Nov 02 11:55:33.721184 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721171    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d239eb0-492f-4335-857e-4edc754823e4-tmp-dir\") pod \"node-resolver-h24nt\" (UID: \"4d239eb0-492f-4335-857e-4edc754823e4\") " pod="openshift-dns/node-resolver-h24nt"
Nov 02 11:55:33.721225 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-kubelet\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.721225 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5ca3e4d6-0eac-4309-b049-e5f597728119-iptables-alerter-script\") pod \"iptables-alerter-dqg86\" (UID: \"5ca3e4d6-0eac-4309-b049-e5f597728119\") " pod="openshift-network-operator/iptables-alerter-dqg86"
Nov 02 11:55:33.721338 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721228    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-hostroot\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.721338 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721242    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-tmp\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.721338 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721321    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-hostroot\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.721338 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721324    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-kubelet\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.721481 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721362    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbnkr\" (UniqueName: \"kubernetes.io/projected/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-kube-api-access-lbnkr\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.721481 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-multus-socket-dir-parent\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.721481 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721398    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a093470-b06b-49c5-9220-2db08ecfde86-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.721481 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721413    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:33.721481 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-slash\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.721676 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721480    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-run-systemd\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.721676 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:33.721497    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 11:55:33.721676 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e81ecb7e-0219-4976-b881-8c87ab61db50-ovn-node-metrics-cert\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.721676 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721526    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d239eb0-492f-4335-857e-4edc754823e4-tmp-dir\") pod \"node-resolver-h24nt\" (UID: \"4d239eb0-492f-4335-857e-4edc754823e4\") " pod="openshift-dns/node-resolver-h24nt"
Nov 02 11:55:33.721676 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721528    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-multus-socket-dir-parent\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.721676 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721535    2580 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 02 11:55:33.721676 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721568    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-sys-fs\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.721676 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721526    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-sys-fs\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.721676 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721578    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-run-systemd\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.721676 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721613    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-slash\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.722085 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:33.721719    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs podName:60dc0a20-c792-4343-aec8-7ff8f86955e9 nodeName:}" failed. No retries permitted until 2026-11-02 11:55:34.221537484 +0000 UTC m=+3.106361760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs") pod "network-metrics-daemon-n4tth" (UID: "60dc0a20-c792-4343-aec8-7ff8f86955e9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 11:55:33.722085 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721764    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7hn7\" (UniqueName: \"kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7\") pod \"network-check-target-2vw9j\" (UID: \"cb734498-5d2a-40c1-9d1f-443e3e8c8f87\") " pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:33.722085 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721781    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5ca3e4d6-0eac-4309-b049-e5f597728119-iptables-alerter-script\") pod \"iptables-alerter-dqg86\" (UID: \"5ca3e4d6-0eac-4309-b049-e5f597728119\") " pod="openshift-network-operator/iptables-alerter-dqg86"
Nov 02 11:55:33.722085 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsnt9\" (UniqueName: \"kubernetes.io/projected/60dc0a20-c792-4343-aec8-7ff8f86955e9-kube-api-access-tsnt9\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:33.722085 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-multus-cni-dir\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.722085 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721838    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc644b68-76cd-4326-8a15-2f8e3dd393d0-host\") pod \"node-ca-xt7dv\" (UID: \"fc644b68-76cd-4326-8a15-2f8e3dd393d0\") " pod="openshift-image-registry/node-ca-xt7dv"
Nov 02 11:55:33.722085 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721858    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ca3e4d6-0eac-4309-b049-e5f597728119-host-slash\") pod \"iptables-alerter-dqg86\" (UID: \"5ca3e4d6-0eac-4309-b049-e5f597728119\") " pod="openshift-network-operator/iptables-alerter-dqg86"
Nov 02 11:55:33.722085 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721879    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb29w\" (UniqueName: \"kubernetes.io/projected/68728c63-ace5-4888-a19b-2782fd2ba673-kube-api-access-hb29w\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.722085 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721898    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-node-log\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.722085 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-run-ovn-kubernetes\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.722085 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721914    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc644b68-76cd-4326-8a15-2f8e3dd393d0-host\") pod \"node-ca-xt7dv\" (UID: \"fc644b68-76cd-4326-8a15-2f8e3dd393d0\") " pod="openshift-image-registry/node-ca-xt7dv"
Nov 02 11:55:33.722085 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721926    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-device-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.722085 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721950    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk6fk\" (UniqueName: \"kubernetes.io/projected/4d239eb0-492f-4335-857e-4edc754823e4-kube-api-access-gk6fk\") pod \"node-resolver-h24nt\" (UID: \"4d239eb0-492f-4335-857e-4edc754823e4\") " pod="openshift-dns/node-resolver-h24nt"
Nov 02 11:55:33.722085 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721958    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-device-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.722085 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-var-lib-cni-bin\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.722085 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.721994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-etc-kubernetes\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.722731 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722010    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a093470-b06b-49c5-9220-2db08ecfde86-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.722731 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722022    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-host\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.722731 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722033    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-node-log\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.722731 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722039    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-registration-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.722731 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722031    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-multus-cni-dir\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.722731 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/68728c63-ace5-4888-a19b-2782fd2ba673-cni-binary-copy\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.722731 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722066    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a093470-b06b-49c5-9220-2db08ecfde86-cni-binary-copy\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.722731 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-var-lib-kubelet\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.722731 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722079    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-run-ovn-kubernetes\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.722731 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722097    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fndj\" (UniqueName: \"kubernetes.io/projected/5ca3e4d6-0eac-4309-b049-e5f597728119-kube-api-access-7fndj\") pod \"iptables-alerter-dqg86\" (UID: \"5ca3e4d6-0eac-4309-b049-e5f597728119\") " pod="openshift-network-operator/iptables-alerter-dqg86"
Nov 02 11:55:33.722731 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fc644b68-76cd-4326-8a15-2f8e3dd393d0-serviceca\") pod \"node-ca-xt7dv\" (UID: \"fc644b68-76cd-4326-8a15-2f8e3dd393d0\") " pod="openshift-image-registry/node-ca-xt7dv"
Nov 02 11:55:33.722731 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722123    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6b350fc9-6bb7-43c4-9b89-c5157f42ca26-agent-certs\") pod \"konnectivity-agent-797nn\" (UID: \"6b350fc9-6bb7-43c4-9b89-c5157f42ca26\") " pod="kube-system/konnectivity-agent-797nn"
Nov 02 11:55:33.722731 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722124    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-var-lib-cni-bin\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.722731 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722125    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-etc-kubernetes\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.722731 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722148    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ca3e4d6-0eac-4309-b049-e5f597728119-host-slash\") pod \"iptables-alerter-dqg86\" (UID: \"5ca3e4d6-0eac-4309-b049-e5f597728119\") " pod="openshift-network-operator/iptables-alerter-dqg86"
Nov 02 11:55:33.722731 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722168    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-run\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.722731 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722183    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-var-lib-kubelet\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.722731 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722188    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-host\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.723385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722190    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-log-socket\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.723385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722229    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-log-socket\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.723385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722240    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-run\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.723385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-kubelet-config\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:33.723385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d239eb0-492f-4335-857e-4edc754823e4-hosts-file\") pod \"node-resolver-h24nt\" (UID: \"4d239eb0-492f-4335-857e-4edc754823e4\") " pod="openshift-dns/node-resolver-h24nt"
Nov 02 11:55:33.723385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722332    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-system-cni-dir\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.723385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722352    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/68728c63-ace5-4888-a19b-2782fd2ba673-multus-daemon-config\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.723385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722369    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-sysctl-conf\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.723385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722387    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-run-openvswitch\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.723385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722400    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d239eb0-492f-4335-857e-4edc754823e4-hosts-file\") pod \"node-resolver-h24nt\" (UID: \"4d239eb0-492f-4335-857e-4edc754823e4\") " pod="openshift-dns/node-resolver-h24nt"
Nov 02 11:55:33.723385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722415    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-cni-netd\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.723385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722450    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-cni-netd\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.723385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722451    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-registration-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.723385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722448    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-kubelet-config\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:33.723385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722505    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-system-cni-dir\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.723385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722512    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-run-openvswitch\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.723385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-dbus\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:33.723385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722544    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-sysctl-conf\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.724029 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722556    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-var-lib-cni-multus\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.724029 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-sys\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.724029 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722585    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-dbus\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:33.724029 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722613    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-run-ovn\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.724029 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722641    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-sys\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.724029 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722644    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a093470-b06b-49c5-9220-2db08ecfde86-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.724029 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722657    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e81ecb7e-0219-4976-b881-8c87ab61db50-env-overrides\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.724029 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722666    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-var-lib-cni-multus\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.724029 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722679    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-run-multus-certs\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.724029 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722688    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-run-ovn\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.724029 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722703    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvpj5\" (UniqueName: \"kubernetes.io/projected/fc644b68-76cd-4326-8a15-2f8e3dd393d0-kube-api-access-tvpj5\") pod \"node-ca-xt7dv\" (UID: \"fc644b68-76cd-4326-8a15-2f8e3dd393d0\") " pod="openshift-image-registry/node-ca-xt7dv"
Nov 02 11:55:33.724029 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722725    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-kubernetes\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.724029 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722734    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-run-multus-certs\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.724029 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722745    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-lib-modules\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.724029 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722767    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g8ls\" (UniqueName: \"kubernetes.io/projected/e81ecb7e-0219-4976-b881-8c87ab61db50-kube-api-access-7g8ls\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.724029 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722784    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-kubernetes\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.724029 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722789    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-etc-selinux\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.724029 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722814    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-cnibin\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.724698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722835    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a093470-b06b-49c5-9220-2db08ecfde86-cnibin\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.724698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722855    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbqbh\" (UniqueName: \"kubernetes.io/projected/0a093470-b06b-49c5-9220-2db08ecfde86-kube-api-access-lbqbh\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.724698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722870    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-lib-modules\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.724698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722878    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-modprobe-d\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.724698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722921    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-cnibin\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.724698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722946    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-systemd\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.724698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722968    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-cni-bin\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.724698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722989    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:33.724698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722996    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a093470-b06b-49c5-9220-2db08ecfde86-cnibin\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.724698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723012    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-run-k8s-cni-cncf-io\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.724698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-run-netns\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.724698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723055    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6b350fc9-6bb7-43c4-9b89-c5157f42ca26-konnectivity-ca\") pod \"konnectivity-agent-797nn\" (UID: \"6b350fc9-6bb7-43c4-9b89-c5157f42ca26\") " pod="kube-system/konnectivity-agent-797nn"
Nov 02 11:55:33.724698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.722967    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-modprobe-d\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.724698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723055    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e81ecb7e-0219-4976-b881-8c87ab61db50-env-overrides\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.724698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723093    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-sysconfig\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.724698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723115    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-etc-openvswitch\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.724698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723135    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e81ecb7e-0219-4976-b881-8c87ab61db50-ovnkube-config\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.725203 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e81ecb7e-0219-4976-b881-8c87ab61db50-ovnkube-script-lib\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.725203 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-socket-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.725203 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723193    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-systemd\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.725203 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723202    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-var-lib-kubelet\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.725203 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723228    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a093470-b06b-49c5-9220-2db08ecfde86-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.725203 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-multus-conf-dir\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.725203 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723256    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/68728c63-ace5-4888-a19b-2782fd2ba673-multus-daemon-config\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.725203 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723271    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-multus-conf-dir\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.725203 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723273    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a093470-b06b-49c5-9220-2db08ecfde86-system-cni-dir\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.725203 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723228    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-cni-bin\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.725203 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-sysctl-d\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.725203 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723332    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fc644b68-76cd-4326-8a15-2f8e3dd393d0-serviceca\") pod \"node-ca-xt7dv\" (UID: \"fc644b68-76cd-4326-8a15-2f8e3dd393d0\") " pod="openshift-image-registry/node-ca-xt7dv"
Nov 02 11:55:33.725203 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723339    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-tuned\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.725203 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:33.723352    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 11:55:33.725203 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723363    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-systemd-units\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.725203 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723385    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a093470-b06b-49c5-9220-2db08ecfde86-system-cni-dir\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.725203 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723386    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-var-lib-openvswitch\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.725689 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:33.723403    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret podName:6d8ba8e4-34a6-47eb-a33f-b67c52acd079 nodeName:}" failed. No retries permitted until 2026-11-02 11:55:34.223388939 +0000 UTC m=+3.108213210 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret") pod "global-pull-secret-syncer-vm62x" (UID: "6d8ba8e4-34a6-47eb-a33f-b67c52acd079") : object "kube-system"/"original-pull-secret" not registered
Nov 02 11:55:33.725689 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723420    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-var-lib-openvswitch\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.725689 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723427    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-etc-openvswitch\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.725689 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723440    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a093470-b06b-49c5-9220-2db08ecfde86-cni-binary-copy\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.725689 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723465    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-run-netns\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.725689 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723471    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-systemd-units\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.725689 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723491    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-socket-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.725689 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723517    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-sysconfig\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.725689 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723564    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-var-lib-kubelet\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.725689 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723153    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-etc-selinux\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.725689 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723633    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbnkn\" (UniqueName: \"kubernetes.io/projected/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-kube-api-access-tbnkn\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.725689 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a093470-b06b-49c5-9220-2db08ecfde86-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.725689 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-run-netns\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.725689 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723706    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.725689 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.725689 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723766    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-sysctl-d\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.726201 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723789    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.726201 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723807    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-run-netns\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.726201 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723825    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.726201 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.723915    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-run-k8s-cni-cncf-io\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.726201 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.724365    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/68728c63-ace5-4888-a19b-2782fd2ba673-cni-binary-copy\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.726201 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.724535    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6b350fc9-6bb7-43c4-9b89-c5157f42ca26-konnectivity-ca\") pod \"konnectivity-agent-797nn\" (UID: \"6b350fc9-6bb7-43c4-9b89-c5157f42ca26\") " pod="kube-system/konnectivity-agent-797nn"
Nov 02 11:55:33.726201 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.724558    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a093470-b06b-49c5-9220-2db08ecfde86-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.726201 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.724835    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e81ecb7e-0219-4976-b881-8c87ab61db50-ovnkube-config\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.726201 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.724887    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e81ecb7e-0219-4976-b881-8c87ab61db50-ovnkube-script-lib\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.726201 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.725572    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-tmp\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.726201 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.725587    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-tuned\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.726201 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.725627    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e81ecb7e-0219-4976-b881-8c87ab61db50-ovn-node-metrics-cert\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.726201 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.725962    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6b350fc9-6bb7-43c4-9b89-c5157f42ca26-agent-certs\") pod \"konnectivity-agent-797nn\" (UID: \"6b350fc9-6bb7-43c4-9b89-c5157f42ca26\") " pod="kube-system/konnectivity-agent-797nn"
Nov 02 11:55:33.775185 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:33.775125    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 11:55:33.775185 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:33.775148    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 11:55:33.775185 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:33.775166    2580 projected.go:196] Error preparing data for projected volume kube-api-access-v7hn7 for pod openshift-network-diagnostics/network-check-target-2vw9j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 11:55:33.775454 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:33.775229    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7 podName:cb734498-5d2a-40c1-9d1f-443e3e8c8f87 nodeName:}" failed. No retries permitted until 2026-11-02 11:55:34.275207511 +0000 UTC m=+3.160031791 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-v7hn7" (UniqueName: "kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7") pod "network-check-target-2vw9j" (UID: "cb734498-5d2a-40c1-9d1f-443e3e8c8f87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 11:55:33.777326 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.777285    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fndj\" (UniqueName: \"kubernetes.io/projected/5ca3e4d6-0eac-4309-b049-e5f597728119-kube-api-access-7fndj\") pod \"iptables-alerter-dqg86\" (UID: \"5ca3e4d6-0eac-4309-b049-e5f597728119\") " pod="openshift-network-operator/iptables-alerter-dqg86"
Nov 02 11:55:33.777414 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.777330    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbnkr\" (UniqueName: \"kubernetes.io/projected/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-kube-api-access-lbnkr\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.777414 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.777404    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk6fk\" (UniqueName: \"kubernetes.io/projected/4d239eb0-492f-4335-857e-4edc754823e4-kube-api-access-gk6fk\") pod \"node-resolver-h24nt\" (UID: \"4d239eb0-492f-4335-857e-4edc754823e4\") " pod="openshift-dns/node-resolver-h24nt"
Nov 02 11:55:33.777513 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.777408    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb29w\" (UniqueName: \"kubernetes.io/projected/68728c63-ace5-4888-a19b-2782fd2ba673-kube-api-access-hb29w\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.777673 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.777626    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsnt9\" (UniqueName: \"kubernetes.io/projected/60dc0a20-c792-4343-aec8-7ff8f86955e9-kube-api-access-tsnt9\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:33.778611 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.778592    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g8ls\" (UniqueName: \"kubernetes.io/projected/e81ecb7e-0219-4976-b881-8c87ab61db50-kube-api-access-7g8ls\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.778716 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.778701    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbnkn\" (UniqueName: \"kubernetes.io/projected/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-kube-api-access-tbnkn\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.779085 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.779071    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbqbh\" (UniqueName: \"kubernetes.io/projected/0a093470-b06b-49c5-9220-2db08ecfde86-kube-api-access-lbqbh\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.779399 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.779386    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvpj5\" (UniqueName: \"kubernetes.io/projected/fc644b68-76cd-4326-8a15-2f8e3dd393d0-kube-api-access-tvpj5\") pod \"node-ca-xt7dv\" (UID: \"fc644b68-76cd-4326-8a15-2f8e3dd393d0\") " pod="openshift-image-registry/node-ca-xt7dv"
Nov 02 11:55:33.901658 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.901630    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-h24nt"
Nov 02 11:55:33.909262 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.909236    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:33.910806 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:33.910786    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d239eb0_492f_4335_857e_4edc754823e4.slice/crio-1626b253692ce32927cd1c75ee25cbd3815299edb3bb10ccc24efaf9075aa336 WatchSource:0}: Error finding container 1626b253692ce32927cd1c75ee25cbd3815299edb3bb10ccc24efaf9075aa336: Status 404 returned error can't find the container with id 1626b253692ce32927cd1c75ee25cbd3815299edb3bb10ccc24efaf9075aa336
Nov 02 11:55:33.912483 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.912466    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fsq7c"
Nov 02 11:55:33.912925 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.912900    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:55:33.917464 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:33.917446    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode81ecb7e_0219_4976_b881_8c87ab61db50.slice/crio-dbd03d2736aff17abe3e92400f910da37ac2f3947dcca36fba592dff611e7de3 WatchSource:0}: Error finding container dbd03d2736aff17abe3e92400f910da37ac2f3947dcca36fba592dff611e7de3: Status 404 returned error can't find the container with id dbd03d2736aff17abe3e92400f910da37ac2f3947dcca36fba592dff611e7de3
Nov 02 11:55:33.919920 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.919905    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xt7dv"
Nov 02 11:55:33.922094 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.922079    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-797nn"
Nov 02 11:55:33.923410 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:33.923332    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68728c63_ace5_4888_a19b_2782fd2ba673.slice/crio-4b5df228477cc031b603768652830fa1d1af537c320e2788ab2e44c2d0663118 WatchSource:0}: Error finding container 4b5df228477cc031b603768652830fa1d1af537c320e2788ab2e44c2d0663118: Status 404 returned error can't find the container with id 4b5df228477cc031b603768652830fa1d1af537c320e2788ab2e44c2d0663118
Nov 02 11:55:33.924942 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:33.924922    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ee9fb0f_d08d_4e09_87bf_df21a5309ca4.slice/crio-7ad8b843416b85c01aba0c78e0dc85ec40f2b20bc8bbc422cd2e3f5f4f0551e9 WatchSource:0}: Error finding container 7ad8b843416b85c01aba0c78e0dc85ec40f2b20bc8bbc422cd2e3f5f4f0551e9: Status 404 returned error can't find the container with id 7ad8b843416b85c01aba0c78e0dc85ec40f2b20bc8bbc422cd2e3f5f4f0551e9
Nov 02 11:55:33.930453 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.930431    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:55:33.931015 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:33.930995    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc644b68_76cd_4326_8a15_2f8e3dd393d0.slice/crio-ba2958d5085f0550a4106550d02a1517e4f0b925a9e2a5a83b34e0bcf8baecb5 WatchSource:0}: Error finding container ba2958d5085f0550a4106550d02a1517e4f0b925a9e2a5a83b34e0bcf8baecb5: Status 404 returned error can't find the container with id ba2958d5085f0550a4106550d02a1517e4f0b925a9e2a5a83b34e0bcf8baecb5
Nov 02 11:55:33.931542 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:33.931523    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b350fc9_6bb7_43c4_9b89_c5157f42ca26.slice/crio-9ce07e1cd2e465b2d044ecc7731bc1dba56885c01c9a992463bd713d620cb571 WatchSource:0}: Error finding container 9ce07e1cd2e465b2d044ecc7731bc1dba56885c01c9a992463bd713d620cb571: Status 404 returned error can't find the container with id 9ce07e1cd2e465b2d044ecc7731bc1dba56885c01c9a992463bd713d620cb571
Nov 02 11:55:33.932441 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.932423    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:55:33.934935 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:33.934920    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dqg86"
Nov 02 11:55:33.940005 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:55:33.939972    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a093470_b06b_49c5_9220_2db08ecfde86.slice/crio-528c27028e7efb73e18821bef8122692542cf1cade92a3a969ea56447b01f550 WatchSource:0}: Error finding container 528c27028e7efb73e18821bef8122692542cf1cade92a3a969ea56447b01f550: Status 404 returned error can't find the container with id 528c27028e7efb73e18821bef8122692542cf1cade92a3a969ea56447b01f550
Nov 02 11:55:34.227369 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:34.227273    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:34.227369 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:34.227354    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:34.227547 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:34.227414    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 11:55:34.227547 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:34.227441    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 11:55:34.227547 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:34.227481    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret podName:6d8ba8e4-34a6-47eb-a33f-b67c52acd079 nodeName:}" failed. No retries permitted until 2026-11-02 11:55:35.227463348 +0000 UTC m=+4.112287620 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret") pod "global-pull-secret-syncer-vm62x" (UID: "6d8ba8e4-34a6-47eb-a33f-b67c52acd079") : object "kube-system"/"original-pull-secret" not registered
Nov 02 11:55:34.227547 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:34.227499    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs podName:60dc0a20-c792-4343-aec8-7ff8f86955e9 nodeName:}" failed. No retries permitted until 2026-11-02 11:55:35.227490413 +0000 UTC m=+4.112314676 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs") pod "network-metrics-daemon-n4tth" (UID: "60dc0a20-c792-4343-aec8-7ff8f86955e9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 11:55:34.327719 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:34.327684    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7hn7\" (UniqueName: \"kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7\") pod \"network-check-target-2vw9j\" (UID: \"cb734498-5d2a-40c1-9d1f-443e3e8c8f87\") " pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:34.327886 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:34.327819    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 11:55:34.327886 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:34.327833    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 11:55:34.327886 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:34.327842    2580 projected.go:196] Error preparing data for projected volume kube-api-access-v7hn7 for pod openshift-network-diagnostics/network-check-target-2vw9j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 11:55:34.327886 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:34.327886    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7 podName:cb734498-5d2a-40c1-9d1f-443e3e8c8f87 nodeName:}" failed. No retries permitted until 2026-11-02 11:55:35.327875057 +0000 UTC m=+4.212699328 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-v7hn7" (UniqueName: "kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7") pod "network-check-target-2vw9j" (UID: "cb734498-5d2a-40c1-9d1f-443e3e8c8f87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 11:55:34.586457 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:34.586367    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dqg86" event={"ID":"5ca3e4d6-0eac-4309-b049-e5f597728119","Type":"ContainerStarted","Data":"c9f277d97a60b0ae286a17629a77187bc220838b3174007f5e7935e23b963dbc"}
Nov 02 11:55:34.587716 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:34.587682    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mwslp" event={"ID":"0a093470-b06b-49c5-9220-2db08ecfde86","Type":"ContainerStarted","Data":"528c27028e7efb73e18821bef8122692542cf1cade92a3a969ea56447b01f550"}
Nov 02 11:55:34.588932 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:34.588903    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xt7dv" event={"ID":"fc644b68-76cd-4326-8a15-2f8e3dd393d0","Type":"ContainerStarted","Data":"ba2958d5085f0550a4106550d02a1517e4f0b925a9e2a5a83b34e0bcf8baecb5"}
Nov 02 11:55:34.590172 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:34.590148    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fsq7c" event={"ID":"68728c63-ace5-4888-a19b-2782fd2ba673","Type":"ContainerStarted","Data":"4b5df228477cc031b603768652830fa1d1af537c320e2788ab2e44c2d0663118"}
Nov 02 11:55:34.591557 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:34.591527    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7686t" event={"ID":"e81ecb7e-0219-4976-b881-8c87ab61db50","Type":"ContainerStarted","Data":"dbd03d2736aff17abe3e92400f910da37ac2f3947dcca36fba592dff611e7de3"}
Nov 02 11:55:34.592826 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:34.592802    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sv47s" event={"ID":"2dc02c96-a4d9-46ab-8b52-5382cf43b92c","Type":"ContainerStarted","Data":"02f5b7ae69a9f8a5f0f18a7906f7ea4ec35c69efb7e84c682bc7fb483b4a2b71"}
Nov 02 11:55:34.594031 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:34.594006    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-797nn" event={"ID":"6b350fc9-6bb7-43c4-9b89-c5157f42ca26","Type":"ContainerStarted","Data":"9ce07e1cd2e465b2d044ecc7731bc1dba56885c01c9a992463bd713d620cb571"}
Nov 02 11:55:34.595193 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:34.595174    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk" event={"ID":"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4","Type":"ContainerStarted","Data":"7ad8b843416b85c01aba0c78e0dc85ec40f2b20bc8bbc422cd2e3f5f4f0551e9"}
Nov 02 11:55:34.596441 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:34.596417    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-h24nt" event={"ID":"4d239eb0-492f-4335-857e-4edc754823e4","Type":"ContainerStarted","Data":"1626b253692ce32927cd1c75ee25cbd3815299edb3bb10ccc24efaf9075aa336"}
Nov 02 11:55:34.671442 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:34.670906    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 11:50:33 +0000 UTC" deadline="2028-07-19 03:10:47.965135716 +0000 UTC"
Nov 02 11:55:34.671442 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:34.670987    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14991h15m13.294156055s"
Nov 02 11:55:35.236208 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:35.235964    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:35.236208 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:35.236035    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:35.236208 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:35.236143    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 11:55:35.236208 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:35.236159    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 11:55:35.236478 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:35.236210    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs podName:60dc0a20-c792-4343-aec8-7ff8f86955e9 nodeName:}" failed. No retries permitted until 2026-11-02 11:55:37.23619097 +0000 UTC m=+6.121015234 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs") pod "network-metrics-daemon-n4tth" (UID: "60dc0a20-c792-4343-aec8-7ff8f86955e9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 11:55:35.236478 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:35.236237    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret podName:6d8ba8e4-34a6-47eb-a33f-b67c52acd079 nodeName:}" failed. No retries permitted until 2026-11-02 11:55:37.236227875 +0000 UTC m=+6.121052134 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret") pod "global-pull-secret-syncer-vm62x" (UID: "6d8ba8e4-34a6-47eb-a33f-b67c52acd079") : object "kube-system"/"original-pull-secret" not registered
Nov 02 11:55:35.336720 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:35.336690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7hn7\" (UniqueName: \"kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7\") pod \"network-check-target-2vw9j\" (UID: \"cb734498-5d2a-40c1-9d1f-443e3e8c8f87\") " pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:35.336876 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:35.336859    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 11:55:35.336933 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:35.336884    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 11:55:35.336933 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:35.336896    2580 projected.go:196] Error preparing data for projected volume kube-api-access-v7hn7 for pod openshift-network-diagnostics/network-check-target-2vw9j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 11:55:35.337095 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:35.336948    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7 podName:cb734498-5d2a-40c1-9d1f-443e3e8c8f87 nodeName:}" failed. No retries permitted until 2026-11-02 11:55:37.336931529 +0000 UTC m=+6.221755803 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-v7hn7" (UniqueName: "kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7") pod "network-check-target-2vw9j" (UID: "cb734498-5d2a-40c1-9d1f-443e3e8c8f87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 11:55:35.586776 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:35.580845    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:35.586776 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:35.580965    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vw9j" podUID="cb734498-5d2a-40c1-9d1f-443e3e8c8f87"
Nov 02 11:55:35.586776 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:35.581041    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:35.586776 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:35.581113    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm62x" podUID="6d8ba8e4-34a6-47eb-a33f-b67c52acd079"
Nov 02 11:55:35.586776 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:35.581171    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:35.586776 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:35.581246    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4tth" podUID="60dc0a20-c792-4343-aec8-7ff8f86955e9"
Nov 02 11:55:35.622742 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:35.621389    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal" event={"ID":"a8a4e5ba356fe85cd903457b215689b7","Type":"ContainerStarted","Data":"1bbc335dc329c0c9c4ca960db29a145dcb80a2d64a01aaa20caedf619ac462b8"}
Nov 02 11:55:35.671742 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:35.671676    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 11:50:33 +0000 UTC" deadline="2028-06-14 16:24:46.315713558 +0000 UTC"
Nov 02 11:55:35.671742 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:35.671739    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14164h29m10.64397834s"
Nov 02 11:55:36.632737 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:36.632435    2580 generic.go:356] "Generic (PLEG): container finished" podID="b0871b91e1bc485b0c3cd650ef3a7622" containerID="5bc8b6704f082f422ad11d5fd02c5c8fabe378e5f98f14db16070b7edad24e65" exitCode=0
Nov 02 11:55:36.633197 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:36.632570    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal" event={"ID":"b0871b91e1bc485b0c3cd650ef3a7622","Type":"ContainerDied","Data":"5bc8b6704f082f422ad11d5fd02c5c8fabe378e5f98f14db16070b7edad24e65"}
Nov 02 11:55:36.750221 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:36.748644    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal" podStartSLOduration=3.7486242560000003 podStartE2EDuration="3.748624256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 11:55:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:55:35.736375369 +0000 UTC m=+4.621199649" watchObservedRunningTime="2026-11-02 11:55:36.748624256 +0000 UTC m=+5.633448537"
Nov 02 11:55:37.254650 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:37.254506    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:37.254650 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:37.254593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:37.254856 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:37.254728    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 11:55:37.254856 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:37.254787    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret podName:6d8ba8e4-34a6-47eb-a33f-b67c52acd079 nodeName:}" failed. No retries permitted until 2026-11-02 11:55:41.254768946 +0000 UTC m=+10.139593220 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret") pod "global-pull-secret-syncer-vm62x" (UID: "6d8ba8e4-34a6-47eb-a33f-b67c52acd079") : object "kube-system"/"original-pull-secret" not registered
Nov 02 11:55:37.254856 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:37.254851    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 11:55:37.255016 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:37.254885    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs podName:60dc0a20-c792-4343-aec8-7ff8f86955e9 nodeName:}" failed. No retries permitted until 2026-11-02 11:55:41.254874109 +0000 UTC m=+10.139698384 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs") pod "network-metrics-daemon-n4tth" (UID: "60dc0a20-c792-4343-aec8-7ff8f86955e9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 11:55:37.355127 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:37.355089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7hn7\" (UniqueName: \"kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7\") pod \"network-check-target-2vw9j\" (UID: \"cb734498-5d2a-40c1-9d1f-443e3e8c8f87\") " pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:37.355745 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:37.355261    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 11:55:37.355745 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:37.355283    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 11:55:37.355745 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:37.355352    2580 projected.go:196] Error preparing data for projected volume kube-api-access-v7hn7 for pod openshift-network-diagnostics/network-check-target-2vw9j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 11:55:37.355745 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:37.355430    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7 podName:cb734498-5d2a-40c1-9d1f-443e3e8c8f87 nodeName:}" failed. No retries permitted until 2026-11-02 11:55:41.355411273 +0000 UTC m=+10.240235547 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-v7hn7" (UniqueName: "kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7") pod "network-check-target-2vw9j" (UID: "cb734498-5d2a-40c1-9d1f-443e3e8c8f87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 11:55:37.578491 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:37.578414    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:37.578673 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:37.578588    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4tth" podUID="60dc0a20-c792-4343-aec8-7ff8f86955e9"
Nov 02 11:55:37.578739 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:37.578706    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:37.578863 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:37.578840    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vw9j" podUID="cb734498-5d2a-40c1-9d1f-443e3e8c8f87"
Nov 02 11:55:37.579375 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:37.579353    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:37.579503 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:37.579456    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm62x" podUID="6d8ba8e4-34a6-47eb-a33f-b67c52acd079"
Nov 02 11:55:37.639789 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:37.639350    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal" event={"ID":"b0871b91e1bc485b0c3cd650ef3a7622","Type":"ContainerStarted","Data":"26287536ef3a282b232fc7ee5b17600e74f1537d737f2b8e643d3253e65fd1fe"}
Nov 02 11:55:37.646381 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:37.644909    2580 generic.go:356] "Generic (PLEG): container finished" podID="0a093470-b06b-49c5-9220-2db08ecfde86" containerID="ec58792d10a1da9b9965785d378713b4f657c9fc0d23da7a3c07d10634c0e946" exitCode=0
Nov 02 11:55:37.646381 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:37.644959    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mwslp" event={"ID":"0a093470-b06b-49c5-9220-2db08ecfde86","Type":"ContainerDied","Data":"ec58792d10a1da9b9965785d378713b4f657c9fc0d23da7a3c07d10634c0e946"}
Nov 02 11:55:37.877589 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:37.877499    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal" podStartSLOduration=4.877482347 podStartE2EDuration="4.877482347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 11:55:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:55:37.755470839 +0000 UTC m=+6.640295125" watchObservedRunningTime="2026-11-02 11:55:37.877482347 +0000 UTC m=+6.762306644"
Nov 02 11:55:38.658416 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:38.658336    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-797nn" event={"ID":"6b350fc9-6bb7-43c4-9b89-c5157f42ca26","Type":"ContainerStarted","Data":"141da7375d73047ff67bae2c8c84cd1bb1e7170dd1d98e8125afb485b8d39f01"}
Nov 02 11:55:38.855984 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:38.855643    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-797nn"
Nov 02 11:55:38.856159 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:38.856144    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-797nn"
Nov 02 11:55:38.970238 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:38.970137    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-797nn" podStartSLOduration=2.733145154 podStartE2EDuration="6.970109112s" totalImagesPullingTime="4.236963958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 11:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:55:38.773454845 +0000 UTC m=+7.658279122" watchObservedRunningTime="2026-11-02 11:55:38.970109112 +0000 UTC m=+7.854933396"
Nov 02 11:55:39.579452 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:39.579411    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:39.579452 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:39.579436    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:39.579654 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:39.579591    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vw9j" podUID="cb734498-5d2a-40c1-9d1f-443e3e8c8f87"
Nov 02 11:55:39.579654 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:39.579641    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:39.579750 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:39.579719    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm62x" podUID="6d8ba8e4-34a6-47eb-a33f-b67c52acd079"
Nov 02 11:55:39.579818 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:39.579794    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4tth" podUID="60dc0a20-c792-4343-aec8-7ff8f86955e9"
Nov 02 11:55:39.664988 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:39.664914    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xt7dv" event={"ID":"fc644b68-76cd-4326-8a15-2f8e3dd393d0","Type":"ContainerStarted","Data":"88f8a92cc77371ade1a976170792643405f38c138105d2a5d3232c759631670d"}
Nov 02 11:55:40.671496 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:40.671367    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk" event={"ID":"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4","Type":"ContainerStarted","Data":"2d0a1d3f397ebec4637c1b1c74a4bc2111eaf80c91bb297f107c7b5bf8c93dfe"}
Nov 02 11:55:40.671496 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:40.671397    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 11:55:41.289574 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:41.289158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:41.289574 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:41.289220    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:41.289574 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:41.289379    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 11:55:41.289574 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:41.289438    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs podName:60dc0a20-c792-4343-aec8-7ff8f86955e9 nodeName:}" failed. No retries permitted until 2026-11-02 11:55:49.289420711 +0000 UTC m=+18.174244984 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs") pod "network-metrics-daemon-n4tth" (UID: "60dc0a20-c792-4343-aec8-7ff8f86955e9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 11:55:41.289574 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:41.289502    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 11:55:41.289574 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:41.289535    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret podName:6d8ba8e4-34a6-47eb-a33f-b67c52acd079 nodeName:}" failed. No retries permitted until 2026-11-02 11:55:49.289524367 +0000 UTC m=+18.174348642 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret") pod "global-pull-secret-syncer-vm62x" (UID: "6d8ba8e4-34a6-47eb-a33f-b67c52acd079") : object "kube-system"/"original-pull-secret" not registered
Nov 02 11:55:41.389879 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:41.389638    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7hn7\" (UniqueName: \"kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7\") pod \"network-check-target-2vw9j\" (UID: \"cb734498-5d2a-40c1-9d1f-443e3e8c8f87\") " pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:41.389879 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:41.389769    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 11:55:41.389879 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:41.389783    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 11:55:41.389879 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:41.389791    2580 projected.go:196] Error preparing data for projected volume kube-api-access-v7hn7 for pod openshift-network-diagnostics/network-check-target-2vw9j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 11:55:41.389879 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:41.389832    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7 podName:cb734498-5d2a-40c1-9d1f-443e3e8c8f87 nodeName:}" failed. No retries permitted until 2026-11-02 11:55:49.389820805 +0000 UTC m=+18.274645077 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-v7hn7" (UniqueName: "kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7") pod "network-check-target-2vw9j" (UID: "cb734498-5d2a-40c1-9d1f-443e3e8c8f87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 11:55:41.581450 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:41.581382    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:41.581450 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:41.581404    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:41.581649 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:41.581490    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vw9j" podUID="cb734498-5d2a-40c1-9d1f-443e3e8c8f87"
Nov 02 11:55:41.581649 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:41.581382    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:41.581649 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:41.581590    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm62x" podUID="6d8ba8e4-34a6-47eb-a33f-b67c52acd079"
Nov 02 11:55:41.581753 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:41.581682    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4tth" podUID="60dc0a20-c792-4343-aec8-7ff8f86955e9"
Nov 02 11:55:42.234345 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:42.234318    2580 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 02 11:55:42.588651 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:42.587840    2580 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-02T11:55:42.234343084Z","UUID":"220d6da3-7a3e-4eaf-b479-a66296cca551","Handler":null,"Name":"","Endpoint":""}
Nov 02 11:55:42.589679 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:42.589619    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 02 11:55:42.589995 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:42.589809    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 02 11:55:42.677917 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:42.677791    2580 generic.go:356] "Generic (PLEG): container finished" podID="0a093470-b06b-49c5-9220-2db08ecfde86" containerID="f8cbdb591d09fc4d9ebe18dea02164a7295dafb868bf816c5db4a9f7c9c9e36e" exitCode=0
Nov 02 11:55:42.677917 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:42.677878    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mwslp" event={"ID":"0a093470-b06b-49c5-9220-2db08ecfde86","Type":"ContainerDied","Data":"f8cbdb591d09fc4d9ebe18dea02164a7295dafb868bf816c5db4a9f7c9c9e36e"}
Nov 02 11:55:42.679947 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:42.679882    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fsq7c" event={"ID":"68728c63-ace5-4888-a19b-2782fd2ba673","Type":"ContainerStarted","Data":"da7b95020f655efdb67917aaa16ffed3168f670062dff6dbcc4f8707a588265c"}
Nov 02 11:55:42.682210 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:42.682187    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk" event={"ID":"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4","Type":"ContainerStarted","Data":"cac7a751d955a3ade7fabd677643a404b7fc1229ecbc20142d0bc4ee4d024889"}
Nov 02 11:55:42.683728 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:42.683708    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-h24nt" event={"ID":"4d239eb0-492f-4335-857e-4edc754823e4","Type":"ContainerStarted","Data":"01f5ab8559779290896ffa78dc61c7b84108a348ec1a1d9988cd39e6cd45403c"}
Nov 02 11:55:42.798263 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:42.798214    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xt7dv" podStartSLOduration=5.383816841 podStartE2EDuration="10.798196966s" totalImagesPullingTime="5.414380125s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 11:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:55:39.779144 +0000 UTC m=+8.663968280" watchObservedRunningTime="2026-11-02 11:55:42.798196966 +0000 UTC m=+11.683021248"
Nov 02 11:55:42.911963 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:42.911707    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-h24nt" podStartSLOduration=3.734472604 podStartE2EDuration="11.911687949s" totalImagesPullingTime="8.177215345s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 11:55:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:55:42.911428743 +0000 UTC m=+11.796253049" watchObservedRunningTime="2026-11-02 11:55:42.911687949 +0000 UTC m=+11.796512231"
Nov 02 11:55:43.579473 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:43.579431    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:43.579896 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:43.579443    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:43.579896 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:43.579559    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm62x" podUID="6d8ba8e4-34a6-47eb-a33f-b67c52acd079"
Nov 02 11:55:43.579896 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:43.579567    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:43.579896 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:43.579652    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vw9j" podUID="cb734498-5d2a-40c1-9d1f-443e3e8c8f87"
Nov 02 11:55:43.579896 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:43.579747    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4tth" podUID="60dc0a20-c792-4343-aec8-7ff8f86955e9"
Nov 02 11:55:44.690000 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:44.689916    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sv47s" event={"ID":"2dc02c96-a4d9-46ab-8b52-5382cf43b92c","Type":"ContainerStarted","Data":"4c98f46c90dfbeeb6e1e067bdfdc2f217b280dcd78942a11d5714e14bb48e934"}
Nov 02 11:55:44.692542 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:44.692507    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk" event={"ID":"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4","Type":"ContainerStarted","Data":"6cd642dfb0af1dfc4230dd788442a9bca896c26a2df465e8f765443494ec7af1"}
Nov 02 11:55:44.694143 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:44.694114    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dqg86" event={"ID":"5ca3e4d6-0eac-4309-b049-e5f597728119","Type":"ContainerStarted","Data":"1d72667ffc01db95924070436ce60f90ce03b6cb1e239e204c8098025056b161"}
Nov 02 11:55:44.696579 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:44.696555    2580 generic.go:356] "Generic (PLEG): container finished" podID="0a093470-b06b-49c5-9220-2db08ecfde86" containerID="ee79522cedf3c4a11e0af63eeb4e9e27033228d8bb535ad52b2845655a286a37" exitCode=0
Nov 02 11:55:44.696698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:44.696592    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mwslp" event={"ID":"0a093470-b06b-49c5-9220-2db08ecfde86","Type":"ContainerDied","Data":"ee79522cedf3c4a11e0af63eeb4e9e27033228d8bb535ad52b2845655a286a37"}
Nov 02 11:55:44.822108 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:44.822056    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-fsq7c" podStartSLOduration=5.923413775 podStartE2EDuration="13.822039862s" totalImagesPullingTime="7.898626087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 11:55:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:55:43.029215364 +0000 UTC m=+11.914039645" watchObservedRunningTime="2026-11-02 11:55:44.822039862 +0000 UTC m=+13.706864140"
Nov 02 11:55:44.822858 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:44.822824    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sv47s" podStartSLOduration=2.396422684 podStartE2EDuration="12.822813863s" totalImagesPullingTime="10.426391179s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 11:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:55:44.822012532 +0000 UTC m=+13.706836812" watchObservedRunningTime="2026-11-02 11:55:44.822813863 +0000 UTC m=+13.707638143"
Nov 02 11:55:44.936230 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:44.936182    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dqg86" podStartSLOduration=4.641398404 podStartE2EDuration="12.936167512s" totalImagesPullingTime="8.294769108s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 11:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:55:44.93602973 +0000 UTC m=+13.820854010" watchObservedRunningTime="2026-11-02 11:55:44.936167512 +0000 UTC m=+13.820991792"
Nov 02 11:55:45.052072 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:45.051973    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk" podStartSLOduration=4.02830947 podStartE2EDuration="14.051958533s" totalImagesPullingTime="10.023649063s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 11:55:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:55:45.051807799 +0000 UTC m=+13.936632081" watchObservedRunningTime="2026-11-02 11:55:45.051958533 +0000 UTC m=+13.936782813"
Nov 02 11:55:45.287477 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:45.287440    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-797nn"
Nov 02 11:55:45.287654 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:45.287583    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 11:55:45.288074 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:45.288057    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-797nn"
Nov 02 11:55:45.578583 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:45.578552    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:45.578583 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:45.578575    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:45.578819 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:45.578684    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vw9j" podUID="cb734498-5d2a-40c1-9d1f-443e3e8c8f87"
Nov 02 11:55:45.578819 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:45.578711    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:45.578819 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:45.578795    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm62x" podUID="6d8ba8e4-34a6-47eb-a33f-b67c52acd079"
Nov 02 11:55:45.578971 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:45.578869    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4tth" podUID="60dc0a20-c792-4343-aec8-7ff8f86955e9"
Nov 02 11:55:46.704033 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:46.703998    2580 generic.go:356] "Generic (PLEG): container finished" podID="0a093470-b06b-49c5-9220-2db08ecfde86" containerID="ca4ccc893748450eb27ab028d9569c57736e88dd48fcc5160aa4cde1cdeb8c7a" exitCode=0
Nov 02 11:55:46.704658 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:46.704036    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mwslp" event={"ID":"0a093470-b06b-49c5-9220-2db08ecfde86","Type":"ContainerDied","Data":"ca4ccc893748450eb27ab028d9569c57736e88dd48fcc5160aa4cde1cdeb8c7a"}
Nov 02 11:55:47.578850 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:47.578674    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:47.579012 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:47.578935    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4tth" podUID="60dc0a20-c792-4343-aec8-7ff8f86955e9"
Nov 02 11:55:47.579251 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:47.579235    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:47.579367 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:47.579317    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vw9j" podUID="cb734498-5d2a-40c1-9d1f-443e3e8c8f87"
Nov 02 11:55:47.579425 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:47.579415    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:47.579500 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:47.579484    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm62x" podUID="6d8ba8e4-34a6-47eb-a33f-b67c52acd079"
Nov 02 11:55:49.357428 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:49.357197    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:49.357428 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:49.357403    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:49.357807 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:49.357345    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 02 11:55:49.357807 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:49.357491    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 11:55:49.357807 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:49.357493    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret podName:6d8ba8e4-34a6-47eb-a33f-b67c52acd079 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:05.357479345 +0000 UTC m=+34.242303617 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret") pod "global-pull-secret-syncer-vm62x" (UID: "6d8ba8e4-34a6-47eb-a33f-b67c52acd079") : object "kube-system"/"original-pull-secret" not registered
Nov 02 11:55:49.357807 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:49.357539    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs podName:60dc0a20-c792-4343-aec8-7ff8f86955e9 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:05.357527922 +0000 UTC m=+34.242352180 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs") pod "network-metrics-daemon-n4tth" (UID: "60dc0a20-c792-4343-aec8-7ff8f86955e9") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 02 11:55:49.458347 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:49.458318    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7hn7\" (UniqueName: \"kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7\") pod \"network-check-target-2vw9j\" (UID: \"cb734498-5d2a-40c1-9d1f-443e3e8c8f87\") " pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:49.458477 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:49.458411    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 02 11:55:49.458477 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:49.458429    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 02 11:55:49.458477 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:49.458437    2580 projected.go:196] Error preparing data for projected volume kube-api-access-v7hn7 for pod openshift-network-diagnostics/network-check-target-2vw9j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 11:55:49.458477 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:49.458476    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7 podName:cb734498-5d2a-40c1-9d1f-443e3e8c8f87 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:05.458465235 +0000 UTC m=+34.343289512 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-v7hn7" (UniqueName: "kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7") pod "network-check-target-2vw9j" (UID: "cb734498-5d2a-40c1-9d1f-443e3e8c8f87") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 02 11:55:49.578382 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:49.578356    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:49.578520 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:49.578401    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:49.578520 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:49.578412    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:49.578520 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:49.578476    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vw9j" podUID="cb734498-5d2a-40c1-9d1f-443e3e8c8f87"
Nov 02 11:55:49.578629 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:49.578578    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm62x" podUID="6d8ba8e4-34a6-47eb-a33f-b67c52acd079"
Nov 02 11:55:49.578714 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:49.578692    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4tth" podUID="60dc0a20-c792-4343-aec8-7ff8f86955e9"
Nov 02 11:55:49.712278 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:49.712203    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7686t" event={"ID":"e81ecb7e-0219-4976-b881-8c87ab61db50","Type":"ContainerStarted","Data":"1db71f2cdbff07495ffc307d24380adf750425b8fed9b31b7007df8bce379850"}
Nov 02 11:55:49.712278 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:49.712273    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7686t" event={"ID":"e81ecb7e-0219-4976-b881-8c87ab61db50","Type":"ContainerStarted","Data":"d3a682a2bda5aac7a9c84ec488c8a97cc3746916f54a55bfc39c66ad534f7769"}
Nov 02 11:55:49.712462 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:49.712320    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7686t" event={"ID":"e81ecb7e-0219-4976-b881-8c87ab61db50","Type":"ContainerStarted","Data":"16d94e9e34943aa62b1a8c5dd87df799b38ca21ce146fa806d64ee9f660b148e"}
Nov 02 11:55:49.712462 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:49.712360    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7686t" event={"ID":"e81ecb7e-0219-4976-b881-8c87ab61db50","Type":"ContainerStarted","Data":"d07a842e8b3f2c4fcc56c755acccca0c657b97413188618f6c7d845bc7023abd"}
Nov 02 11:55:49.712462 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:49.712390    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7686t" event={"ID":"e81ecb7e-0219-4976-b881-8c87ab61db50","Type":"ContainerStarted","Data":"c79f94434122e7c50738b7b3928df476ea18ef3a61d488e6668888b56f5e388d"}
Nov 02 11:55:49.712462 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:49.712441    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7686t" event={"ID":"e81ecb7e-0219-4976-b881-8c87ab61db50","Type":"ContainerStarted","Data":"4b8976d70250a0aad58180510c8e6a4b2182ccd3756340952f54cf3c8a57cba9"}
Nov 02 11:55:49.714879 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:49.714851    2580 generic.go:356] "Generic (PLEG): container finished" podID="0a093470-b06b-49c5-9220-2db08ecfde86" containerID="2fdcbee48ff8fe9f4325e088e7e69be0830fd72dd191e52e20066f6170e6972b" exitCode=0
Nov 02 11:55:49.714999 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:49.714883    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mwslp" event={"ID":"0a093470-b06b-49c5-9220-2db08ecfde86","Type":"ContainerDied","Data":"2fdcbee48ff8fe9f4325e088e7e69be0830fd72dd191e52e20066f6170e6972b"}
Nov 02 11:55:50.719707 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:50.719674    2580 generic.go:356] "Generic (PLEG): container finished" podID="0a093470-b06b-49c5-9220-2db08ecfde86" containerID="48e67b64d1ff53132026a68e8a6ecd5539096a8660ed95b7014b827bed3bee71" exitCode=0
Nov 02 11:55:50.720154 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:50.719718    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mwslp" event={"ID":"0a093470-b06b-49c5-9220-2db08ecfde86","Type":"ContainerDied","Data":"48e67b64d1ff53132026a68e8a6ecd5539096a8660ed95b7014b827bed3bee71"}
Nov 02 11:55:51.579535 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:51.579456    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:51.579684 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:51.579542    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4tth" podUID="60dc0a20-c792-4343-aec8-7ff8f86955e9"
Nov 02 11:55:51.579684 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:51.579641    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:51.579800 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:51.579745    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm62x" podUID="6d8ba8e4-34a6-47eb-a33f-b67c52acd079"
Nov 02 11:55:51.579854 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:51.579802    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:51.579928 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:51.579913    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vw9j" podUID="cb734498-5d2a-40c1-9d1f-443e3e8c8f87"
Nov 02 11:55:51.724428 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:51.724393    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mwslp" event={"ID":"0a093470-b06b-49c5-9220-2db08ecfde86","Type":"ContainerStarted","Data":"3fac484e04b26020d26c73b01d76a76fd1f708f8353be5fb40921ee6c2df88da"}
Nov 02 11:55:51.727451 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:51.727422    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7686t" event={"ID":"e81ecb7e-0219-4976-b881-8c87ab61db50","Type":"ContainerStarted","Data":"dade84603fca809ba398e220a9bc7d50ad0c32195537c7966c80e2e976ab1674"}
Nov 02 11:55:51.860434 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:51.860393    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mwslp" podStartSLOduration=8.094720447 podStartE2EDuration="20.860381486s" totalImagesPullingTime="12.765661039s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 11:55:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:55:51.859767758 +0000 UTC m=+20.744592039" watchObservedRunningTime="2026-11-02 11:55:51.860381486 +0000 UTC m=+20.745205767"
Nov 02 11:55:53.581733 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:53.581503    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:53.582374 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:53.581503    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:53.582374 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:53.581788    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4tth" podUID="60dc0a20-c792-4343-aec8-7ff8f86955e9"
Nov 02 11:55:53.582374 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:53.581520    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:53.582374 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:53.581838    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm62x" podUID="6d8ba8e4-34a6-47eb-a33f-b67c52acd079"
Nov 02 11:55:53.582374 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:53.581900    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vw9j" podUID="cb734498-5d2a-40c1-9d1f-443e3e8c8f87"
Nov 02 11:55:53.735285 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:53.735254    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7686t" event={"ID":"e81ecb7e-0219-4976-b881-8c87ab61db50","Type":"ContainerStarted","Data":"2315af4d7b80060bb6e49135af44f68c869f95f683aebd7856683e77141195cd"}
Nov 02 11:55:53.735582 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:53.735556    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:53.735703 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:53.735627    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:53.735703 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:53.735680    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:53.750719 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:53.750691    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:53.750865 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:53.750835    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:55:54.009464 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:54.009409    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7686t" podStartSLOduration=8.186498171 podStartE2EDuration="23.009396931s" totalImagesPullingTime="14.82289876s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 11:55:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:55:53.959697976 +0000 UTC m=+22.844522258" watchObservedRunningTime="2026-11-02 11:55:54.009396931 +0000 UTC m=+22.894221211"
Nov 02 11:55:55.578264 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:55.578233    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:55.578778 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:55.578290    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:55.578778 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:55.578314    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:55.578778 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:55.578425    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vw9j" podUID="cb734498-5d2a-40c1-9d1f-443e3e8c8f87"
Nov 02 11:55:55.578778 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:55.578501    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm62x" podUID="6d8ba8e4-34a6-47eb-a33f-b67c52acd079"
Nov 02 11:55:55.578778 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:55.578607    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4tth" podUID="60dc0a20-c792-4343-aec8-7ff8f86955e9"
Nov 02 11:55:55.945823 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:55.945787    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2vw9j"]
Nov 02 11:55:55.945967 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:55.945909    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:55.946045 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:55.946021    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vw9j" podUID="cb734498-5d2a-40c1-9d1f-443e3e8c8f87"
Nov 02 11:55:55.949567 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:55.949542    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vm62x"]
Nov 02 11:55:55.949683 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:55.949588    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n4tth"]
Nov 02 11:55:55.949683 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:55.949638    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:55.949775 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:55.949731    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4tth" podUID="60dc0a20-c792-4343-aec8-7ff8f86955e9"
Nov 02 11:55:55.949816 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:55.949787    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:55.949897 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:55.949878    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm62x" podUID="6d8ba8e4-34a6-47eb-a33f-b67c52acd079"
Nov 02 11:55:57.578634 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:57.578568    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:57.578634 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:57.578585    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:57.579135 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:57.578568    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:57.579135 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:57.578670    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm62x" podUID="6d8ba8e4-34a6-47eb-a33f-b67c52acd079"
Nov 02 11:55:57.579232 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:57.579181    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4tth" podUID="60dc0a20-c792-4343-aec8-7ff8f86955e9"
Nov 02 11:55:57.579424 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:57.579398    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vw9j" podUID="cb734498-5d2a-40c1-9d1f-443e3e8c8f87"
Nov 02 11:55:59.578459 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:59.578384    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:55:59.578459 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:59.578436    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:55:59.578459 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:55:59.578456    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:55:59.579039 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:59.578558    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm62x" podUID="6d8ba8e4-34a6-47eb-a33f-b67c52acd079"
Nov 02 11:55:59.579039 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:59.578609    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4tth" podUID="60dc0a20-c792-4343-aec8-7ff8f86955e9"
Nov 02 11:55:59.579039 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:55:59.578706    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vw9j" podUID="cb734498-5d2a-40c1-9d1f-443e3e8c8f87"
Nov 02 11:56:01.579281 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:01.579224    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:56:01.579630 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:01.579321    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vw9j" podUID="cb734498-5d2a-40c1-9d1f-443e3e8c8f87"
Nov 02 11:56:01.579630 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:01.579378    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:56:01.579630 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:01.579473    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vm62x" podUID="6d8ba8e4-34a6-47eb-a33f-b67c52acd079"
Nov 02 11:56:01.579630 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:01.579512    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:56:01.579630 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:01.579567    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n4tth" podUID="60dc0a20-c792-4343-aec8-7ff8f86955e9"
Nov 02 11:56:01.932798 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:01.932624    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeReady"
Nov 02 11:56:01.933016 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:01.932915    2580 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 02 11:56:02.046329 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.046271    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-sgrg9"]
Nov 02 11:56:02.053350 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.053328    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-sgrg9"
Nov 02 11:56:02.095855 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.095831    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-lskz4"]
Nov 02 11:56:02.100082 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.100064    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-7tt7c"]
Nov 02 11:56:02.100201 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.100175    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-lskz4"
Nov 02 11:56:02.102987 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.102968    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d58f4b759-ts6c2"]
Nov 02 11:56:02.103108 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.103089    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-7tt7c"
Nov 02 11:56:02.105784 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.105761    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"]
Nov 02 11:56:02.105884 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.105870    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.108911 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.108894    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-rhl9j"]
Nov 02 11:56:02.109005 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.108991    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:02.111950 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.111935    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-567988bb57-rzsqv"]
Nov 02 11:56:02.112050 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.112037    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.114730 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.114714    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"]
Nov 02 11:56:02.114839 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.114824    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:02.117513 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.117427    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-2qq64"]
Nov 02 11:56:02.117632 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.117547    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:02.120464 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.120444    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"]
Nov 02 11:56:02.120569 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.120552    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:02.123159 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.123136    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"
Nov 02 11:56:02.145196 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.145061    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 02 11:56:02.145826 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.145774    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 02 11:56:02.146926 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.146908    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"]
Nov 02 11:56:02.149799 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.149780    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9pfhc"]
Nov 02 11:56:02.149951 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.149930    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"
Nov 02 11:56:02.153744 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.153728    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-g7gnd"]
Nov 02 11:56:02.153851 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.153823    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9pfhc"
Nov 02 11:56:02.156796 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.156773    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-hflkz"]
Nov 02 11:56:02.156884 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.156871    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-g7gnd"
Nov 02 11:56:02.158191 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.158175    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 02 11:56:02.158191 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.158174    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 02 11:56:02.158360 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.158238    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 02 11:56:02.158360 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.158315    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7p526\""
Nov 02 11:56:02.159469 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.159453    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75766bbfd6-gpgxx"]
Nov 02 11:56:02.159543 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.159530    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:02.160805 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.160762    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 02 11:56:02.162575 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.162558    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tlvw4"]
Nov 02 11:56:02.162703 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.162686    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.164583 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.164560    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r554w\" (UniqueName: \"kubernetes.io/projected/5e041a46-74d9-4fc1-82e7-6c83d2b3c229-kube-api-access-r554w\") pod \"kas-connection-checker-685895c7c9-sgrg9\" (UID: \"5e041a46-74d9-4fc1-82e7-6c83d2b3c229\") " pod="kube-system/kas-connection-checker-685895c7c9-sgrg9"
Nov 02 11:56:02.165167 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.165153    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cttl7"]
Nov 02 11:56:02.165293 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.165275    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tlvw4"
Nov 02 11:56:02.167978 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.167964    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-lskz4"]
Nov 02 11:56:02.168046 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.167989    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-sgrg9"]
Nov 02 11:56:02.168046 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.168010    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-7tt7c"]
Nov 02 11:56:02.168046 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.168034    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d58f4b759-ts6c2"]
Nov 02 11:56:02.168133 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.168087    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:02.187528 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.187506    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-g7gnd"]
Nov 02 11:56:02.192527 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.192505    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"]
Nov 02 11:56:02.193322 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.193276    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"]
Nov 02 11:56:02.195537 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.195336    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-rhl9j"]
Nov 02 11:56:02.196242 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.196223    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-2qq64"]
Nov 02 11:56:02.197008 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.196989    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-567988bb57-rzsqv"]
Nov 02 11:56:02.205347 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.205324    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cttl7"]
Nov 02 11:56:02.211908 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.211884    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 02 11:56:02.212063 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.212048    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 02 11:56:02.212821 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.212798    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 02 11:56:02.212821 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.212811    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 02 11:56:02.212981 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.212833    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 02 11:56:02.212981 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.212911    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 02 11:56:02.212981 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.212929    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 02 11:56:02.212981 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.212944    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 02 11:56:02.212981 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.212967    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 02 11:56:02.213200 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.213035    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 02 11:56:02.213200 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.213080    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 02 11:56:02.213285 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.213212    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 02 11:56:02.213356 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.213337    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 02 11:56:02.213411 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.213357    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 02 11:56:02.213459 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.213404    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 02 11:56:02.213459 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.213362    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 02 11:56:02.213459 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.213434    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 02 11:56:02.213625 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.213479    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 02 11:56:02.213625 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.213529    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 02 11:56:02.213625 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.213601    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 02 11:56:02.213740 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.213729    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 02 11:56:02.213740 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.213735    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 02 11:56:02.213829 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.213741    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 02 11:56:02.213829 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.213774    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 02 11:56:02.260721 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.260686    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 02 11:56:02.265231 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265205    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b54rp\" (UniqueName: \"kubernetes.io/projected/9176bf1e-471e-43a2-af6b-650c5a494efa-kube-api-access-b54rp\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.265355 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-image-registry-private-configuration\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.265405 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265379    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:02.265440 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265405    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/81050700-fe2d-45db-b66e-c83400cb2d35-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:02.265440 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265425    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j7r5\" (UniqueName: \"kubernetes.io/projected/13d85b4b-e889-48b8-97af-f23632533bea-kube-api-access-7j7r5\") pod \"ingress-canary-tlvw4\" (UID: \"13d85b4b-e889-48b8-97af-f23632533bea\") " pod="openshift-ingress-canary/ingress-canary-tlvw4"
Nov 02 11:56:02.265513 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265445    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:02.265513 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265467    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:02.265583 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265535    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-certificates\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.265583 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265556    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:02.265583 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265573    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-ca-trust-extracted\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.265700 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265587    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9176bf1e-471e-43a2-af6b-650c5a494efa-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.265700 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265604    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-stats-auth\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:02.265759 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265712    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-trusted-ca\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:02.265759 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265750    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zstf\" (UniqueName: \"kubernetes.io/projected/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-kube-api-access-6zstf\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:02.265845 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265774    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-certificates\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.265888 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265836    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-default-certificate\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:02.265888 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265882    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmwmm\" (UniqueName: \"kubernetes.io/projected/15291745-62ec-45f2-85f9-373504d79de2-kube-api-access-xmwmm\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:02.265968 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265901    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-bound-sa-token\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.265968 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265924    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-config-volume\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:02.266069 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.265967    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-trusted-ca\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.266069 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266026    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfp9f\" (UniqueName: \"kubernetes.io/projected/269be49a-6e81-4730-960e-cb0b4a7b031b-kube-api-access-hfp9f\") pod \"kas-connection-checker-685895c7c9-lskz4\" (UID: \"269be49a-6e81-4730-960e-cb0b4a7b031b\") " pod="kube-system/kas-connection-checker-685895c7c9-lskz4"
Nov 02 11:56:02.266069 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266059    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/691a1fbe-dd29-4579-8e72-8c36c2b1eda2-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf\" (UID: \"691a1fbe-dd29-4579-8e72-8c36c2b1eda2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"
Nov 02 11:56:02.266157 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266114    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9nlh\" (UniqueName: \"kubernetes.io/projected/f38c9faa-5fae-45ba-93f6-e252b6f092fd-kube-api-access-g9nlh\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:02.266191 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13d85b4b-e889-48b8-97af-f23632533bea-cert\") pod \"ingress-canary-tlvw4\" (UID: \"13d85b4b-e889-48b8-97af-f23632533bea\") " pod="openshift-ingress-canary/ingress-canary-tlvw4"
Nov 02 11:56:02.266235 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266185    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/691a1fbe-dd29-4579-8e72-8c36c2b1eda2-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf\" (UID: \"691a1fbe-dd29-4579-8e72-8c36c2b1eda2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"
Nov 02 11:56:02.266287 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266240    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-serving-cert\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:02.266356 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266320    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-image-registry-private-configuration\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.266410 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266354    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqzcg\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-kube-api-access-vqzcg\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.266410 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266384    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.266489 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266433    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9176bf1e-471e-43a2-af6b-650c5a494efa-serving-cert\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.266547 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-trusted-ca\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.266600 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266570    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hxmh\" (UniqueName: \"kubernetes.io/projected/a05c4de6-e34a-416d-9ed9-35fa6a614dcc-kube-api-access-7hxmh\") pod \"volume-data-source-validator-58c9f5f46d-9pfhc\" (UID: \"a05c4de6-e34a-416d-9ed9-35fa6a614dcc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9pfhc"
Nov 02 11:56:02.266707 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266600    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwnz6\" (UniqueName: \"kubernetes.io/projected/cc29f105-14d5-4004-afe0-1d0c8c17a5fb-kube-api-access-nwnz6\") pod \"kas-connection-checker-685895c7c9-7tt7c\" (UID: \"cc29f105-14d5-4004-afe0-1d0c8c17a5fb\") " pod="kube-system/kas-connection-checker-685895c7c9-7tt7c"
Nov 02 11:56:02.266707 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266690    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-tmp-dir\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:02.266787 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266768    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-installation-pull-secrets\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.266856 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266841    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6384f50d-3bbe-47ef-8e56-5a72d8113c6f-config\") pod \"service-ca-operator-544d749c8c-9mfg4\" (UID: \"6384f50d-3bbe-47ef-8e56-5a72d8113c6f\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"
Nov 02 11:56:02.266898 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266870    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-bound-sa-token\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.266978 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266961    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:02.267022 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.266997    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9176bf1e-471e-43a2-af6b-650c5a494efa-snapshots\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.267065 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.267022    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:02.267065 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.267060    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xjgq\" (UniqueName: \"kubernetes.io/projected/98301ff8-eb60-43d4-857f-da7d8da3858b-kube-api-access-2xjgq\") pod \"network-check-source-6f78d59f99-g7gnd\" (UID: \"98301ff8-eb60-43d4-857f-da7d8da3858b\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-g7gnd"
Nov 02 11:56:02.267152 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.267080    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-config\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:02.267206 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.267180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r554w\" (UniqueName: \"kubernetes.io/projected/5e041a46-74d9-4fc1-82e7-6c83d2b3c229-kube-api-access-r554w\") pod \"kas-connection-checker-685895c7c9-sgrg9\" (UID: \"5e041a46-74d9-4fc1-82e7-6c83d2b3c229\") " pod="kube-system/kas-connection-checker-685895c7c9-sgrg9"
Nov 02 11:56:02.267249 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.267211    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6384f50d-3bbe-47ef-8e56-5a72d8113c6f-serving-cert\") pod \"service-ca-operator-544d749c8c-9mfg4\" (UID: \"6384f50d-3bbe-47ef-8e56-5a72d8113c6f\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"
Nov 02 11:56:02.267287 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.267247    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6jj5\" (UniqueName: \"kubernetes.io/projected/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-kube-api-access-d6jj5\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:02.267287 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.267270    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-installation-pull-secrets\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.267388 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.267287    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2759fb5a-17ad-4c29-b9c8-175769f53069-ca-trust-extracted\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.267388 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.267326    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mg2c\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-kube-api-access-5mg2c\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.267388 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.267346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9176bf1e-471e-43a2-af6b-650c5a494efa-tmp\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.267527 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.267380    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9176bf1e-471e-43a2-af6b-650c5a494efa-service-ca-bundle\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.267527 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.267428    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-nginx-conf\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:02.267527 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.267487    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.267664 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.267535    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v4hm\" (UniqueName: \"kubernetes.io/projected/6384f50d-3bbe-47ef-8e56-5a72d8113c6f-kube-api-access-8v4hm\") pod \"service-ca-operator-544d749c8c-9mfg4\" (UID: \"6384f50d-3bbe-47ef-8e56-5a72d8113c6f\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"
Nov 02 11:56:02.267664 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.267579    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgndt\" (UniqueName: \"kubernetes.io/projected/81050700-fe2d-45db-b66e-c83400cb2d35-kube-api-access-xgndt\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:02.267664 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.267607    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:02.267664 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.267633    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdzs7\" (UniqueName: \"kubernetes.io/projected/691a1fbe-dd29-4579-8e72-8c36c2b1eda2-kube-api-access-vdzs7\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf\" (UID: \"691a1fbe-dd29-4579-8e72-8c36c2b1eda2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"
Nov 02 11:56:02.288737 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.288713    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 02 11:56:02.291776 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.291761    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 02 11:56:02.293805 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.293779    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 02 11:56:02.293917 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.293832    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 02 11:56:02.293917 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.293850    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 02 11:56:02.293917 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.293901    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 02 11:56:02.294236 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.294216    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 02 11:56:02.294654 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.294638    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tlvw4"]
Nov 02 11:56:02.294981 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.294967    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 02 11:56:02.295047 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.295027    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 02 11:56:02.295096 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.295056    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 02 11:56:02.295096 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.295066    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 02 11:56:02.295187 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.295059    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 02 11:56:02.295187 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.295109    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 02 11:56:02.295187 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.295111    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 02 11:56:02.295187 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.295125    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qcv25\""
Nov 02 11:56:02.295187 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.295128    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 02 11:56:02.295187 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.295159    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-msdx6\""
Nov 02 11:56:02.295385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.295258    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 02 11:56:02.295385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.295314    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 02 11:56:02.295385 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.295363    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-hqqr9\""
Nov 02 11:56:02.295463 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.295418    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 02 11:56:02.295802 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.295788    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"]
Nov 02 11:56:02.296396 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.296385    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-hflkz"]
Nov 02 11:56:02.297002 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.296990    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"]
Nov 02 11:56:02.297595 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.297578    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9pfhc"]
Nov 02 11:56:02.304075 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.304060    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75766bbfd6-gpgxx"]
Nov 02 11:56:02.328945 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.328927    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r554w\" (UniqueName: \"kubernetes.io/projected/5e041a46-74d9-4fc1-82e7-6c83d2b3c229-kube-api-access-r554w\") pod \"kas-connection-checker-685895c7c9-sgrg9\" (UID: \"5e041a46-74d9-4fc1-82e7-6c83d2b3c229\") " pod="kube-system/kas-connection-checker-685895c7c9-sgrg9"
Nov 02 11:56:02.364780 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.364755    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-sgrg9"
Nov 02 11:56:02.368197 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-installation-pull-secrets\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.368307 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6384f50d-3bbe-47ef-8e56-5a72d8113c6f-config\") pod \"service-ca-operator-544d749c8c-9mfg4\" (UID: \"6384f50d-3bbe-47ef-8e56-5a72d8113c6f\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"
Nov 02 11:56:02.368307 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-bound-sa-token\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.368307 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368239    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:02.368307 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9176bf1e-471e-43a2-af6b-650c5a494efa-snapshots\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.368307 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368279    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:02.368549 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368322    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xjgq\" (UniqueName: \"kubernetes.io/projected/98301ff8-eb60-43d4-857f-da7d8da3858b-kube-api-access-2xjgq\") pod \"network-check-source-6f78d59f99-g7gnd\" (UID: \"98301ff8-eb60-43d4-857f-da7d8da3858b\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-g7gnd"
Nov 02 11:56:02.368549 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368347    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-config\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:02.368549 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368395    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6384f50d-3bbe-47ef-8e56-5a72d8113c6f-serving-cert\") pod \"service-ca-operator-544d749c8c-9mfg4\" (UID: \"6384f50d-3bbe-47ef-8e56-5a72d8113c6f\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"
Nov 02 11:56:02.368549 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.368412    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:02.868393142 +0000 UTC m=+31.753217422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : configmap references non-existent config key: service-ca.crt
Nov 02 11:56:02.368760 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6jj5\" (UniqueName: \"kubernetes.io/projected/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-kube-api-access-d6jj5\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:02.368760 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368622    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-installation-pull-secrets\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.368760 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2759fb5a-17ad-4c29-b9c8-175769f53069-ca-trust-extracted\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.368760 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368675    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mg2c\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-kube-api-access-5mg2c\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.368760 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9176bf1e-471e-43a2-af6b-650c5a494efa-tmp\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.369005 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368765    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9176bf1e-471e-43a2-af6b-650c5a494efa-service-ca-bundle\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.369005 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368788    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-nginx-conf\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:02.369005 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368819    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.369005 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v4hm\" (UniqueName: \"kubernetes.io/projected/6384f50d-3bbe-47ef-8e56-5a72d8113c6f-kube-api-access-8v4hm\") pod \"service-ca-operator-544d749c8c-9mfg4\" (UID: \"6384f50d-3bbe-47ef-8e56-5a72d8113c6f\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"
Nov 02 11:56:02.369005 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368871    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgndt\" (UniqueName: \"kubernetes.io/projected/81050700-fe2d-45db-b66e-c83400cb2d35-kube-api-access-xgndt\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:02.369005 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368896    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:02.369005 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368912    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6384f50d-3bbe-47ef-8e56-5a72d8113c6f-config\") pod \"service-ca-operator-544d749c8c-9mfg4\" (UID: \"6384f50d-3bbe-47ef-8e56-5a72d8113c6f\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"
Nov 02 11:56:02.369005 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368922    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdzs7\" (UniqueName: \"kubernetes.io/projected/691a1fbe-dd29-4579-8e72-8c36c2b1eda2-kube-api-access-vdzs7\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf\" (UID: \"691a1fbe-dd29-4579-8e72-8c36c2b1eda2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"
Nov 02 11:56:02.369005 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368959    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b54rp\" (UniqueName: \"kubernetes.io/projected/9176bf1e-471e-43a2-af6b-650c5a494efa-kube-api-access-b54rp\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.369005 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368950    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9176bf1e-471e-43a2-af6b-650c5a494efa-snapshots\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.369005 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.368986    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-image-registry-private-configuration\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.369519 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.369024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:02.369519 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.369078    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/81050700-fe2d-45db-b66e-c83400cb2d35-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:02.369519 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.369107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j7r5\" (UniqueName: \"kubernetes.io/projected/13d85b4b-e889-48b8-97af-f23632533bea-kube-api-access-7j7r5\") pod \"ingress-canary-tlvw4\" (UID: \"13d85b4b-e889-48b8-97af-f23632533bea\") " pod="openshift-ingress-canary/ingress-canary-tlvw4"
Nov 02 11:56:02.369519 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.369136    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:02.369519 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.369161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:02.369519 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.369199    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-certificates\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.369519 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.369252    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:02.369519 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.369324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-ca-trust-extracted\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.369519 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.369352    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9176bf1e-471e-43a2-af6b-650c5a494efa-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.369519 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.369367    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9176bf1e-471e-43a2-af6b-650c5a494efa-tmp\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.369519 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.369378    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-config\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:02.369519 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.369386    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2759fb5a-17ad-4c29-b9c8-175769f53069-ca-trust-extracted\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.369519 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.369408    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-stats-auth\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:02.369519 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.369449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-trusted-ca\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:02.369519 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.369482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zstf\" (UniqueName: \"kubernetes.io/projected/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-kube-api-access-6zstf\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:02.369519 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.369512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-certificates\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.370388 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.369550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-default-certificate\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:02.370388 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.369581    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmwmm\" (UniqueName: \"kubernetes.io/projected/15291745-62ec-45f2-85f9-373504d79de2-kube-api-access-xmwmm\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:02.370388 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.369652    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 11:56:02.370388 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.369701    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls podName:bbfab1b5-14db-477a-8f06-7cf2c1d058a2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:02.869684917 +0000 UTC m=+31.754509180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls") pod "dns-default-cttl7" (UID: "bbfab1b5-14db-477a-8f06-7cf2c1d058a2") : secret "dns-default-metrics-tls" not found
Nov 02 11:56:02.370388 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.369755    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 11:56:02.370388 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.369788    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert podName:c0e3ecc0-e47b-4612-87aa-53ce8434f5a8 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:02.869778067 +0000 UTC m=+31.754602325 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-hflkz" (UID: "c0e3ecc0-e47b-4612-87aa-53ce8434f5a8") : secret "networking-console-plugin-cert" not found
Nov 02 11:56:02.370825 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.370704    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/81050700-fe2d-45db-b66e-c83400cb2d35-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:02.370896 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.370839    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9176bf1e-471e-43a2-af6b-650c5a494efa-service-ca-bundle\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.371560 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.371021    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 11:56:02.371560 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.371072    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls podName:f38c9faa-5fae-45ba-93f6-e252b6f092fd nodeName:}" failed. No retries permitted until 2026-11-02 11:56:02.871057486 +0000 UTC m=+31.755881750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-9vksm" (UID: "f38c9faa-5fae-45ba-93f6-e252b6f092fd") : secret "samples-operator-tls" not found
Nov 02 11:56:02.371560 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-bound-sa-token\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.371560 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371133    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-config-volume\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:02.371560 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371162    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-trusted-ca\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.371560 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371196    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfp9f\" (UniqueName: \"kubernetes.io/projected/269be49a-6e81-4730-960e-cb0b4a7b031b-kube-api-access-hfp9f\") pod \"kas-connection-checker-685895c7c9-lskz4\" (UID: \"269be49a-6e81-4730-960e-cb0b4a7b031b\") " pod="kube-system/kas-connection-checker-685895c7c9-lskz4"
Nov 02 11:56:02.371560 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371225    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/691a1fbe-dd29-4579-8e72-8c36c2b1eda2-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf\" (UID: \"691a1fbe-dd29-4579-8e72-8c36c2b1eda2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"
Nov 02 11:56:02.371560 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371257    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9nlh\" (UniqueName: \"kubernetes.io/projected/f38c9faa-5fae-45ba-93f6-e252b6f092fd-kube-api-access-g9nlh\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:02.371560 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371288    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13d85b4b-e889-48b8-97af-f23632533bea-cert\") pod \"ingress-canary-tlvw4\" (UID: \"13d85b4b-e889-48b8-97af-f23632533bea\") " pod="openshift-ingress-canary/ingress-canary-tlvw4"
Nov 02 11:56:02.371560 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/691a1fbe-dd29-4579-8e72-8c36c2b1eda2-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf\" (UID: \"691a1fbe-dd29-4579-8e72-8c36c2b1eda2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"
Nov 02 11:56:02.371560 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371367    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-serving-cert\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:02.371560 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-image-registry-private-configuration\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.371560 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371414    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-installation-pull-secrets\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.371560 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqzcg\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-kube-api-access-vqzcg\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.371560 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371507    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.371560 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371532    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9176bf1e-471e-43a2-af6b-650c5a494efa-serving-cert\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.372322 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371535    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-nginx-conf\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:02.372322 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-trusted-ca\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.372322 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371605    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hxmh\" (UniqueName: \"kubernetes.io/projected/a05c4de6-e34a-416d-9ed9-35fa6a614dcc-kube-api-access-7hxmh\") pod \"volume-data-source-validator-58c9f5f46d-9pfhc\" (UID: \"a05c4de6-e34a-416d-9ed9-35fa6a614dcc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9pfhc"
Nov 02 11:56:02.372322 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.371632    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 11:56:02.372322 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.371647    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75766bbfd6-gpgxx: secret "image-registry-tls" not found
Nov 02 11:56:02.372322 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.371688    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls podName:2759fb5a-17ad-4c29-b9c8-175769f53069 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:02.87167413 +0000 UTC m=+31.756498390 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls") pod "image-registry-75766bbfd6-gpgxx" (UID: "2759fb5a-17ad-4c29-b9c8-175769f53069") : secret "image-registry-tls" not found
Nov 02 11:56:02.372322 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.371711    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 11:56:02.372322 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.371726    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 11:56:02.372322 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.371762    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls podName:81050700-fe2d-45db-b66e-c83400cb2d35 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:02.871746544 +0000 UTC m=+31.756570811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2dm77" (UID: "81050700-fe2d-45db-b66e-c83400cb2d35") : secret "cluster-monitoring-operator-tls" not found
Nov 02 11:56:02.372322 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.371784    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:02.871769587 +0000 UTC m=+31.756593889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : secret "router-metrics-certs-default" not found
Nov 02 11:56:02.372322 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371631    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwnz6\" (UniqueName: \"kubernetes.io/projected/cc29f105-14d5-4004-afe0-1d0c8c17a5fb-kube-api-access-nwnz6\") pod \"kas-connection-checker-685895c7c9-7tt7c\" (UID: \"cc29f105-14d5-4004-afe0-1d0c8c17a5fb\") " pod="kube-system/kas-connection-checker-685895c7c9-7tt7c"
Nov 02 11:56:02.372322 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.371962    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-tmp-dir\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:02.372322 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.372061    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6384f50d-3bbe-47ef-8e56-5a72d8113c6f-serving-cert\") pod \"service-ca-operator-544d749c8c-9mfg4\" (UID: \"6384f50d-3bbe-47ef-8e56-5a72d8113c6f\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"
Nov 02 11:56:02.372322 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.372269    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-certificates\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.373028 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.372380    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 11:56:02.373028 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.372394    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d58f4b759-ts6c2: secret "image-registry-tls" not found
Nov 02 11:56:02.373028 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.372398    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-tmp-dir\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:02.373028 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.372431    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls podName:e9dc2cae-7e99-4282-a02d-51fb0f12b43b nodeName:}" failed. No retries permitted until 2026-11-02 11:56:02.872417035 +0000 UTC m=+31.757241295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls") pod "image-registry-6d58f4b759-ts6c2" (UID: "e9dc2cae-7e99-4282-a02d-51fb0f12b43b") : secret "image-registry-tls" not found
Nov 02 11:56:02.373028 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.372434    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-config-volume\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:02.373028 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.372743    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-ca-trust-extracted\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.373360 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.373087    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-certificates\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.373360 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.373242    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9176bf1e-471e-43a2-af6b-650c5a494efa-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.373360 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.373255    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-trusted-ca\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:02.375431 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.374052    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/691a1fbe-dd29-4579-8e72-8c36c2b1eda2-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf\" (UID: \"691a1fbe-dd29-4579-8e72-8c36c2b1eda2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"
Nov 02 11:56:02.375431 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.374482    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:02.375431 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.374737    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-image-registry-private-configuration\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.375431 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.374754    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-installation-pull-secrets\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.375431 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.375132    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-trusted-ca\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.375431 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.375143    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9176bf1e-471e-43a2-af6b-650c5a494efa-serving-cert\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.375431 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.375183    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-stats-auth\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:02.375431 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.375352    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/691a1fbe-dd29-4579-8e72-8c36c2b1eda2-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf\" (UID: \"691a1fbe-dd29-4579-8e72-8c36c2b1eda2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"
Nov 02 11:56:02.375431 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.375363    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-image-registry-private-configuration\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.375796 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.375507    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-trusted-ca\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.375972 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.375947    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-default-certificate\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:02.377147 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.377117    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-serving-cert\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:02.377252 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.377208    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13d85b4b-e889-48b8-97af-f23632533bea-cert\") pod \"ingress-canary-tlvw4\" (UID: \"13d85b4b-e889-48b8-97af-f23632533bea\") " pod="openshift-ingress-canary/ingress-canary-tlvw4"
Nov 02 11:56:02.432291 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.432262    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xjgq\" (UniqueName: \"kubernetes.io/projected/98301ff8-eb60-43d4-857f-da7d8da3858b-kube-api-access-2xjgq\") pod \"network-check-source-6f78d59f99-g7gnd\" (UID: \"98301ff8-eb60-43d4-857f-da7d8da3858b\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-g7gnd"
Nov 02 11:56:02.432291 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.432287    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6jj5\" (UniqueName: \"kubernetes.io/projected/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-kube-api-access-d6jj5\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:02.432481 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.432440    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-bound-sa-token\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.436014 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.435959    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mg2c\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-kube-api-access-5mg2c\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.436185 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.436163    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j7r5\" (UniqueName: \"kubernetes.io/projected/13d85b4b-e889-48b8-97af-f23632533bea-kube-api-access-7j7r5\") pod \"ingress-canary-tlvw4\" (UID: \"13d85b4b-e889-48b8-97af-f23632533bea\") " pod="openshift-ingress-canary/ingress-canary-tlvw4"
Nov 02 11:56:02.436424 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.436405    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmwmm\" (UniqueName: \"kubernetes.io/projected/15291745-62ec-45f2-85f9-373504d79de2-kube-api-access-xmwmm\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:02.437359 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.437336    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwnz6\" (UniqueName: \"kubernetes.io/projected/cc29f105-14d5-4004-afe0-1d0c8c17a5fb-kube-api-access-nwnz6\") pod \"kas-connection-checker-685895c7c9-7tt7c\" (UID: \"cc29f105-14d5-4004-afe0-1d0c8c17a5fb\") " pod="kube-system/kas-connection-checker-685895c7c9-7tt7c"
Nov 02 11:56:02.437509 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.437491    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdzs7\" (UniqueName: \"kubernetes.io/projected/691a1fbe-dd29-4579-8e72-8c36c2b1eda2-kube-api-access-vdzs7\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf\" (UID: \"691a1fbe-dd29-4579-8e72-8c36c2b1eda2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"
Nov 02 11:56:02.438047 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.438022    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgndt\" (UniqueName: \"kubernetes.io/projected/81050700-fe2d-45db-b66e-c83400cb2d35-kube-api-access-xgndt\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:02.439956 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.439890    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hxmh\" (UniqueName: \"kubernetes.io/projected/a05c4de6-e34a-416d-9ed9-35fa6a614dcc-kube-api-access-7hxmh\") pod \"volume-data-source-validator-58c9f5f46d-9pfhc\" (UID: \"a05c4de6-e34a-416d-9ed9-35fa6a614dcc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9pfhc"
Nov 02 11:56:02.439956 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.439924    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfp9f\" (UniqueName: \"kubernetes.io/projected/269be49a-6e81-4730-960e-cb0b4a7b031b-kube-api-access-hfp9f\") pod \"kas-connection-checker-685895c7c9-lskz4\" (UID: \"269be49a-6e81-4730-960e-cb0b4a7b031b\") " pod="kube-system/kas-connection-checker-685895c7c9-lskz4"
Nov 02 11:56:02.439956 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.439934    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zstf\" (UniqueName: \"kubernetes.io/projected/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-kube-api-access-6zstf\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:02.441001 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.440977    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b54rp\" (UniqueName: \"kubernetes.io/projected/9176bf1e-471e-43a2-af6b-650c5a494efa-kube-api-access-b54rp\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.441120 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.441106    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v4hm\" (UniqueName: \"kubernetes.io/projected/6384f50d-3bbe-47ef-8e56-5a72d8113c6f-kube-api-access-8v4hm\") pod \"service-ca-operator-544d749c8c-9mfg4\" (UID: \"6384f50d-3bbe-47ef-8e56-5a72d8113c6f\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"
Nov 02 11:56:02.441398 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.441374    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9nlh\" (UniqueName: \"kubernetes.io/projected/f38c9faa-5fae-45ba-93f6-e252b6f092fd-kube-api-access-g9nlh\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:02.441509 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.441496    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqzcg\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-kube-api-access-vqzcg\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.442361 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.442342    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-bound-sa-token\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.460802 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.460518    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:02.466589 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.466568    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"
Nov 02 11:56:02.472613 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.472589    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"
Nov 02 11:56:02.476036 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.475819    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9pfhc"
Nov 02 11:56:02.505123 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.505086    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-g7gnd"
Nov 02 11:56:02.549837 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.549807    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tlvw4"
Nov 02 11:56:02.711799 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.711742    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-lskz4"
Nov 02 11:56:02.718910 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.718881    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-7tt7c"
Nov 02 11:56:02.740476 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.740455    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:02.786167 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.786137    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-sgrg9"]
Nov 02 11:56:02.791176 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:56:02.791013    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e041a46_74d9_4fc1_82e7_6c83d2b3c229.slice/crio-157531b7c6576657c1682cfa3945f3764b167240c374c1029cdb9badcf0b03e7 WatchSource:0}: Error finding container 157531b7c6576657c1682cfa3945f3764b167240c374c1029cdb9badcf0b03e7: Status 404 returned error can't find the container with id 157531b7c6576657c1682cfa3945f3764b167240c374c1029cdb9badcf0b03e7
Nov 02 11:56:02.861742 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.861712    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-2qq64"]
Nov 02 11:56:02.868426 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:56:02.868395    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53ea0d03_1f13_4ee1_af04_9ddf449d6b33.slice/crio-e6f5f644265046944ccd3504362891bab45948af07c5ceb4eb229f945beb2a60 WatchSource:0}: Error finding container e6f5f644265046944ccd3504362891bab45948af07c5ceb4eb229f945beb2a60: Status 404 returned error can't find the container with id e6f5f644265046944ccd3504362891bab45948af07c5ceb4eb229f945beb2a60
Nov 02 11:56:02.876584 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.876553    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:02.876670 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.876630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:02.876670 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.876661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:02.876782 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.876733    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:02.876782 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.876768    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:02.876897 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.876807    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:02.876897 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.876832    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:02.876897 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.876869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:02.877042 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.876921    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 11:56:02.877042 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.876960    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 11:56:02.877042 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.876973    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:03.876955739 +0000 UTC m=+32.761780021 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : secret "router-metrics-certs-default" not found
Nov 02 11:56:02.877042 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.876993    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls podName:81050700-fe2d-45db-b66e-c83400cb2d35 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:03.876983176 +0000 UTC m=+32.761807449 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2dm77" (UID: "81050700-fe2d-45db-b66e-c83400cb2d35") : secret "cluster-monitoring-operator-tls" not found
Nov 02 11:56:02.877042 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.877040    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 11:56:02.877286 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.877051    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d58f4b759-ts6c2: secret "image-registry-tls" not found
Nov 02 11:56:02.877286 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.877082    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls podName:e9dc2cae-7e99-4282-a02d-51fb0f12b43b nodeName:}" failed. No retries permitted until 2026-11-02 11:56:03.877072691 +0000 UTC m=+32.761896969 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls") pod "image-registry-6d58f4b759-ts6c2" (UID: "e9dc2cae-7e99-4282-a02d-51fb0f12b43b") : secret "image-registry-tls" not found
Nov 02 11:56:02.877286 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.877088    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 11:56:02.877286 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.877117    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert podName:c0e3ecc0-e47b-4612-87aa-53ce8434f5a8 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:03.877108432 +0000 UTC m=+32.761932708 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-hflkz" (UID: "c0e3ecc0-e47b-4612-87aa-53ce8434f5a8") : secret "networking-console-plugin-cert" not found
Nov 02 11:56:02.877286 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.877145    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:03.877135316 +0000 UTC m=+32.761959590 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : configmap references non-existent config key: service-ca.crt
Nov 02 11:56:02.877286 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.877043    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 11:56:02.877286 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.877162    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75766bbfd6-gpgxx: secret "image-registry-tls" not found
Nov 02 11:56:02.877286 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.877167    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 11:56:02.877286 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.877189    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls podName:2759fb5a-17ad-4c29-b9c8-175769f53069 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:03.877180476 +0000 UTC m=+32.762004752 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls") pod "image-registry-75766bbfd6-gpgxx" (UID: "2759fb5a-17ad-4c29-b9c8-175769f53069") : secret "image-registry-tls" not found
Nov 02 11:56:02.877286 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.877204    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls podName:f38c9faa-5fae-45ba-93f6-e252b6f092fd nodeName:}" failed. No retries permitted until 2026-11-02 11:56:03.87719574 +0000 UTC m=+32.762019998 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-9vksm" (UID: "f38c9faa-5fae-45ba-93f6-e252b6f092fd") : secret "samples-operator-tls" not found
Nov 02 11:56:02.877286 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.877230    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 11:56:02.877286 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:02.877248    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls podName:bbfab1b5-14db-477a-8f06-7cf2c1d058a2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:03.87724271 +0000 UTC m=+32.762066968 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls") pod "dns-default-cttl7" (UID: "bbfab1b5-14db-477a-8f06-7cf2c1d058a2") : secret "dns-default-metrics-tls" not found
Nov 02 11:56:02.890393 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.890365    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"]
Nov 02 11:56:02.895849 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:56:02.895816    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6384f50d_3bbe_47ef_8e56_5a72d8113c6f.slice/crio-ec3be01ca9e36f5d00ffc5c704ed09be987f8ca357bc7a3e448d221dba99b078 WatchSource:0}: Error finding container ec3be01ca9e36f5d00ffc5c704ed09be987f8ca357bc7a3e448d221dba99b078: Status 404 returned error can't find the container with id ec3be01ca9e36f5d00ffc5c704ed09be987f8ca357bc7a3e448d221dba99b078
Nov 02 11:56:02.918235 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.918213    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-g7gnd"]
Nov 02 11:56:02.920469 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:56:02.920441    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98301ff8_eb60_43d4_857f_da7d8da3858b.slice/crio-a129b2ee37d0619b409638f2e04aad33408dd2dd73f19fec71d9c40bd039ad35 WatchSource:0}: Error finding container a129b2ee37d0619b409638f2e04aad33408dd2dd73f19fec71d9c40bd039ad35: Status 404 returned error can't find the container with id a129b2ee37d0619b409638f2e04aad33408dd2dd73f19fec71d9c40bd039ad35
Nov 02 11:56:02.956522 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.956501    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"]
Nov 02 11:56:02.959007 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:56:02.958985    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod691a1fbe_dd29_4579_8e72_8c36c2b1eda2.slice/crio-a2ccfcfa40dba291334b84b4bc17c12052b8b1d5979211e1e6778c097be27411 WatchSource:0}: Error finding container a2ccfcfa40dba291334b84b4bc17c12052b8b1d5979211e1e6778c097be27411: Status 404 returned error can't find the container with id a2ccfcfa40dba291334b84b4bc17c12052b8b1d5979211e1e6778c097be27411
Nov 02 11:56:02.974526 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:02.974484    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9pfhc"]
Nov 02 11:56:02.976843 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:56:02.976823    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda05c4de6_e34a_416d_9ed9_35fa6a614dcc.slice/crio-2fad6c8bb3e025df2fcd1778e6bc1e7ad6d715f97a65b8e02e6ca4c254806413 WatchSource:0}: Error finding container 2fad6c8bb3e025df2fcd1778e6bc1e7ad6d715f97a65b8e02e6ca4c254806413: Status 404 returned error can't find the container with id 2fad6c8bb3e025df2fcd1778e6bc1e7ad6d715f97a65b8e02e6ca4c254806413
Nov 02 11:56:03.001195 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.001172    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tlvw4"]
Nov 02 11:56:03.003810 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:56:03.003781    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13d85b4b_e889_48b8_97af_f23632533bea.slice/crio-2f3ecd16e39a81d34aa0291317a80fa6fdeeff614be213fcf4c761a24ee92b4e WatchSource:0}: Error finding container 2f3ecd16e39a81d34aa0291317a80fa6fdeeff614be213fcf4c761a24ee92b4e: Status 404 returned error can't find the container with id 2f3ecd16e39a81d34aa0291317a80fa6fdeeff614be213fcf4c761a24ee92b4e
Nov 02 11:56:03.109949 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.109924    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-lskz4"]
Nov 02 11:56:03.112541 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:56:03.112514    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod269be49a_6e81_4730_960e_cb0b4a7b031b.slice/crio-7717a04a35c612622fe1587f3cf2f5d814bab7717ee54f48a2e4a319a83546d5 WatchSource:0}: Error finding container 7717a04a35c612622fe1587f3cf2f5d814bab7717ee54f48a2e4a319a83546d5: Status 404 returned error can't find the container with id 7717a04a35c612622fe1587f3cf2f5d814bab7717ee54f48a2e4a319a83546d5
Nov 02 11:56:03.153824 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.153666    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-7tt7c"]
Nov 02 11:56:03.155925 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:56:03.155898    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc29f105_14d5_4004_afe0_1d0c8c17a5fb.slice/crio-8d4366f1cd075adb9f1bdcdbdcff5bd0278d670a602795381004153b9933001b WatchSource:0}: Error finding container 8d4366f1cd075adb9f1bdcdbdcff5bd0278d670a602795381004153b9933001b: Status 404 returned error can't find the container with id 8d4366f1cd075adb9f1bdcdbdcff5bd0278d670a602795381004153b9933001b
Nov 02 11:56:03.182922 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.182897    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-rhl9j"]
Nov 02 11:56:03.195941 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:56:03.195913    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9176bf1e_471e_43a2_af6b_650c5a494efa.slice/crio-4710cea14069f1ce66928eff09f9ff1fa2ab600549eeb4737a1075940c93d2f0 WatchSource:0}: Error finding container 4710cea14069f1ce66928eff09f9ff1fa2ab600549eeb4737a1075940c93d2f0: Status 404 returned error can't find the container with id 4710cea14069f1ce66928eff09f9ff1fa2ab600549eeb4737a1075940c93d2f0
Nov 02 11:56:03.578572 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.578498    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:56:03.578723 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.578503    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:56:03.578723 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.578710    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:56:03.630967 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.630942    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 02 11:56:03.631101 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.631081    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gfx2g\""
Nov 02 11:56:03.631141 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.631081    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 02 11:56:03.631141 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.631091    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-lk8z8\""
Nov 02 11:56:03.631141 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.631122    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-hc67r\""
Nov 02 11:56:03.758353 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.758325    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tlvw4" event={"ID":"13d85b4b-e889-48b8-97af-f23632533bea","Type":"ContainerStarted","Data":"2f3ecd16e39a81d34aa0291317a80fa6fdeeff614be213fcf4c761a24ee92b4e"}
Nov 02 11:56:03.759349 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.759318    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-g7gnd" event={"ID":"98301ff8-eb60-43d4-857f-da7d8da3858b","Type":"ContainerStarted","Data":"a129b2ee37d0619b409638f2e04aad33408dd2dd73f19fec71d9c40bd039ad35"}
Nov 02 11:56:03.760292 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.760271    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4" event={"ID":"6384f50d-3bbe-47ef-8e56-5a72d8113c6f","Type":"ContainerStarted","Data":"ec3be01ca9e36f5d00ffc5c704ed09be987f8ca357bc7a3e448d221dba99b078"}
Nov 02 11:56:03.761275 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.761251    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64" event={"ID":"53ea0d03-1f13-4ee1-af04-9ddf449d6b33","Type":"ContainerStarted","Data":"e6f5f644265046944ccd3504362891bab45948af07c5ceb4eb229f945beb2a60"}
Nov 02 11:56:03.762332 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.762311    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-rhl9j" event={"ID":"9176bf1e-471e-43a2-af6b-650c5a494efa","Type":"ContainerStarted","Data":"4710cea14069f1ce66928eff09f9ff1fa2ab600549eeb4737a1075940c93d2f0"}
Nov 02 11:56:03.763335 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.763316    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9pfhc" event={"ID":"a05c4de6-e34a-416d-9ed9-35fa6a614dcc","Type":"ContainerStarted","Data":"2fad6c8bb3e025df2fcd1778e6bc1e7ad6d715f97a65b8e02e6ca4c254806413"}
Nov 02 11:56:03.764543 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.764525    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-sgrg9" event={"ID":"5e041a46-74d9-4fc1-82e7-6c83d2b3c229","Type":"ContainerStarted","Data":"4d6165337bcfd1ab3c20fa12eb10f6756133211c549ade747eab27952a0ed9a3"}
Nov 02 11:56:03.764621 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.764558    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-sgrg9" event={"ID":"5e041a46-74d9-4fc1-82e7-6c83d2b3c229","Type":"ContainerStarted","Data":"157531b7c6576657c1682cfa3945f3764b167240c374c1029cdb9badcf0b03e7"}
Nov 02 11:56:03.765866 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.765847    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-7tt7c" event={"ID":"cc29f105-14d5-4004-afe0-1d0c8c17a5fb","Type":"ContainerStarted","Data":"c54e1a5219dfdb96caa1d805f9f47a28fc45845051543534e6393c6d8da5e18d"}
Nov 02 11:56:03.765961 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.765889    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-7tt7c" event={"ID":"cc29f105-14d5-4004-afe0-1d0c8c17a5fb","Type":"ContainerStarted","Data":"8d4366f1cd075adb9f1bdcdbdcff5bd0278d670a602795381004153b9933001b"}
Nov 02 11:56:03.767125 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.767107    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-lskz4" event={"ID":"269be49a-6e81-4730-960e-cb0b4a7b031b","Type":"ContainerStarted","Data":"9865230b623c323a50e9bb6df9b8a278a08bb29f08b6f24e644fe47f7e6a52f5"}
Nov 02 11:56:03.767201 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.767146    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-lskz4" event={"ID":"269be49a-6e81-4730-960e-cb0b4a7b031b","Type":"ContainerStarted","Data":"7717a04a35c612622fe1587f3cf2f5d814bab7717ee54f48a2e4a319a83546d5"}
Nov 02 11:56:03.768143 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.768124    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf" event={"ID":"691a1fbe-dd29-4579-8e72-8c36c2b1eda2","Type":"ContainerStarted","Data":"a2ccfcfa40dba291334b84b4bc17c12052b8b1d5979211e1e6778c097be27411"}
Nov 02 11:56:03.888688 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.888663    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:03.888811 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.888702    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:03.888811 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.888789    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:03.888911 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:03.888829    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:05.888810568 +0000 UTC m=+34.773634834 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : configmap references non-existent config key: service-ca.crt
Nov 02 11:56:03.888911 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:03.888879    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 11:56:03.888911 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.888881    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:03.889052 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:03.888939    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:05.888924185 +0000 UTC m=+34.773748455 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : secret "router-metrics-certs-default" not found
Nov 02 11:56:03.889052 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.888933    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:03.889052 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.888970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:03.889052 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.889010    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:03.889052 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:03.888883    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 11:56:03.889052 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:03.889043    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75766bbfd6-gpgxx: secret "image-registry-tls" not found
Nov 02 11:56:03.889335 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:03.889071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:03.889335 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:03.889076    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 11:56:03.889335 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:03.889080    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 11:56:03.889335 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:03.888951    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 11:56:03.889335 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:03.889103    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls podName:2759fb5a-17ad-4c29-b9c8-175769f53069 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:05.889092676 +0000 UTC m=+34.773916936 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls") pod "image-registry-75766bbfd6-gpgxx" (UID: "2759fb5a-17ad-4c29-b9c8-175769f53069") : secret "image-registry-tls" not found
Nov 02 11:56:03.889335 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:03.888993    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 11:56:03.889335 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:03.889127    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls podName:bbfab1b5-14db-477a-8f06-7cf2c1d058a2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:05.889115835 +0000 UTC m=+34.773940098 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls") pod "dns-default-cttl7" (UID: "bbfab1b5-14db-477a-8f06-7cf2c1d058a2") : secret "dns-default-metrics-tls" not found
Nov 02 11:56:03.889335 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:03.889141    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls podName:81050700-fe2d-45db-b66e-c83400cb2d35 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:05.889135028 +0000 UTC m=+34.773959286 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2dm77" (UID: "81050700-fe2d-45db-b66e-c83400cb2d35") : secret "cluster-monitoring-operator-tls" not found
Nov 02 11:56:03.889335 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:03.889151    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert podName:c0e3ecc0-e47b-4612-87aa-53ce8434f5a8 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:05.889146208 +0000 UTC m=+34.773970466 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-hflkz" (UID: "c0e3ecc0-e47b-4612-87aa-53ce8434f5a8") : secret "networking-console-plugin-cert" not found
Nov 02 11:56:03.889335 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:03.889181    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 11:56:03.889335 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:03.889191    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d58f4b759-ts6c2: secret "image-registry-tls" not found
Nov 02 11:56:03.889335 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:03.889198    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls podName:f38c9faa-5fae-45ba-93f6-e252b6f092fd nodeName:}" failed. No retries permitted until 2026-11-02 11:56:05.889192875 +0000 UTC m=+34.774017133 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-9vksm" (UID: "f38c9faa-5fae-45ba-93f6-e252b6f092fd") : secret "samples-operator-tls" not found
Nov 02 11:56:03.889335 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:03.889220    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls podName:e9dc2cae-7e99-4282-a02d-51fb0f12b43b nodeName:}" failed. No retries permitted until 2026-11-02 11:56:05.889207151 +0000 UTC m=+34.774031423 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls") pod "image-registry-6d58f4b759-ts6c2" (UID: "e9dc2cae-7e99-4282-a02d-51fb0f12b43b") : secret "image-registry-tls" not found
Nov 02 11:56:04.209719 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:04.209608    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-lskz4" podStartSLOduration=395.209589149 podStartE2EDuration="6m35.209589149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 11:49:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:56:04.208400218 +0000 UTC m=+33.093224499" watchObservedRunningTime="2026-11-02 11:56:04.209589149 +0000 UTC m=+33.094413427"
Nov 02 11:56:04.324328 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:04.324253    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-7tt7c" podStartSLOduration=395.32423329 podStartE2EDuration="6m35.32423329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 11:49:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:56:04.322614457 +0000 UTC m=+33.207438739" watchObservedRunningTime="2026-11-02 11:56:04.32423329 +0000 UTC m=+33.209057573"
Nov 02 11:56:04.438864 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:04.438779    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-sgrg9" podStartSLOduration=395.438760585 podStartE2EDuration="6m35.438760585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 11:49:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:56:04.437543318 +0000 UTC m=+33.322367600" watchObservedRunningTime="2026-11-02 11:56:04.438760585 +0000 UTC m=+33.323584868"
Nov 02 11:56:05.406755 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:05.406702    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:56:05.407285 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:05.407066    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:56:05.407285 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.407226    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 11:56:05.407424 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.407315    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs podName:60dc0a20-c792-4343-aec8-7ff8f86955e9 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:37.407278294 +0000 UTC m=+66.292102575 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs") pod "network-metrics-daemon-n4tth" (UID: "60dc0a20-c792-4343-aec8-7ff8f86955e9") : secret "metrics-daemon-secret" not found
Nov 02 11:56:05.414330 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:05.414289    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:56:05.508176 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:05.508139    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7hn7\" (UniqueName: \"kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7\") pod \"network-check-target-2vw9j\" (UID: \"cb734498-5d2a-40c1-9d1f-443e3e8c8f87\") " pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:56:05.512231 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:05.512186    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7hn7\" (UniqueName: \"kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7\") pod \"network-check-target-2vw9j\" (UID: \"cb734498-5d2a-40c1-9d1f-443e3e8c8f87\") " pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:56:05.708704 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:05.708663    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:56:05.747629 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:05.747543    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:56:05.785617 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:05.785585    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9pfhc" event={"ID":"a05c4de6-e34a-416d-9ed9-35fa6a614dcc","Type":"ContainerStarted","Data":"b3ac30d017b8e95ab5221f08ab9bef3b3714fdba7cc1053841aa7d11009159ef"}
Nov 02 11:56:05.913370 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:05.913336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:05.913524 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:05.913397    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:05.913524 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:05.913452    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:05.913524 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:05.913481    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:05.913524 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:05.913519    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:05.913658 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:05.913629    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:05.913698 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:05.913676    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:05.913745 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:05.913706    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:05.913897 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.913879    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 11:56:05.913957 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.913951    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:09.913930879 +0000 UTC m=+38.798755158 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : secret "router-metrics-certs-default" not found
Nov 02 11:56:05.914106 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.914089    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 11:56:05.914181 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.914119    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75766bbfd6-gpgxx: secret "image-registry-tls" not found
Nov 02 11:56:05.914181 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.914163    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls podName:2759fb5a-17ad-4c29-b9c8-175769f53069 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:09.914149615 +0000 UTC m=+38.798973921 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls") pod "image-registry-75766bbfd6-gpgxx" (UID: "2759fb5a-17ad-4c29-b9c8-175769f53069") : secret "image-registry-tls" not found
Nov 02 11:56:05.914275 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.914233    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 11:56:05.914275 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.914271    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert podName:c0e3ecc0-e47b-4612-87aa-53ce8434f5a8 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:09.914259114 +0000 UTC m=+38.799083375 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-hflkz" (UID: "c0e3ecc0-e47b-4612-87aa-53ce8434f5a8") : secret "networking-console-plugin-cert" not found
Nov 02 11:56:05.914388 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.914347    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 11:56:05.914388 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.914377    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls podName:f38c9faa-5fae-45ba-93f6-e252b6f092fd nodeName:}" failed. No retries permitted until 2026-11-02 11:56:09.914367612 +0000 UTC m=+38.799191872 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-9vksm" (UID: "f38c9faa-5fae-45ba-93f6-e252b6f092fd") : secret "samples-operator-tls" not found
Nov 02 11:56:05.914490 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.914446    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 11:56:05.914490 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.914465    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d58f4b759-ts6c2: secret "image-registry-tls" not found
Nov 02 11:56:05.914580 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.914518    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls podName:e9dc2cae-7e99-4282-a02d-51fb0f12b43b nodeName:}" failed. No retries permitted until 2026-11-02 11:56:09.914499519 +0000 UTC m=+38.799323780 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls") pod "image-registry-6d58f4b759-ts6c2" (UID: "e9dc2cae-7e99-4282-a02d-51fb0f12b43b") : secret "image-registry-tls" not found
Nov 02 11:56:05.914632 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.914588    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 11:56:05.914632 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.914623    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls podName:bbfab1b5-14db-477a-8f06-7cf2c1d058a2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:09.914612784 +0000 UTC m=+38.799437044 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls") pod "dns-default-cttl7" (UID: "bbfab1b5-14db-477a-8f06-7cf2c1d058a2") : secret "dns-default-metrics-tls" not found
Nov 02 11:56:05.914730 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.914680    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 11:56:05.914730 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.914711    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls podName:81050700-fe2d-45db-b66e-c83400cb2d35 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:09.914700832 +0000 UTC m=+38.799525094 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2dm77" (UID: "81050700-fe2d-45db-b66e-c83400cb2d35") : secret "cluster-monitoring-operator-tls" not found
Nov 02 11:56:05.914825 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:05.914788    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:09.91476264 +0000 UTC m=+38.799586900 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : configmap references non-existent config key: service-ca.crt
Nov 02 11:56:06.158523 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:06.158446    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vm62x"]
Nov 02 11:56:06.212437 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:06.212409    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2vw9j"]
Nov 02 11:56:06.220235 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:56:06.220207    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb734498_5d2a_40c1_9d1f_443e3e8c8f87.slice/crio-d33d00be8285b37a83e9f995f598d1de1e53c29edb532922ea2642f91786578b WatchSource:0}: Error finding container d33d00be8285b37a83e9f995f598d1de1e53c29edb532922ea2642f91786578b: Status 404 returned error can't find the container with id d33d00be8285b37a83e9f995f598d1de1e53c29edb532922ea2642f91786578b
Nov 02 11:56:06.799745 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:06.799703    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vm62x" event={"ID":"6d8ba8e4-34a6-47eb-a33f-b67c52acd079","Type":"ContainerStarted","Data":"300e91d7e2bd6fd9552f384c619772601ce569e29f65a2c676b716953c72a443"}
Nov 02 11:56:06.802591 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:06.802363    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2vw9j" event={"ID":"cb734498-5d2a-40c1-9d1f-443e3e8c8f87","Type":"ContainerStarted","Data":"d33d00be8285b37a83e9f995f598d1de1e53c29edb532922ea2642f91786578b"}
Nov 02 11:56:06.919939 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:06.918607    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9pfhc" podStartSLOduration=368.296084377 podStartE2EDuration="6m10.91858787s" totalImagesPullingTime="2.622503493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 11:49:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:56:06.917778025 +0000 UTC m=+35.802602307" watchObservedRunningTime="2026-11-02 11:56:06.91858787 +0000 UTC m=+35.803412152"
Nov 02 11:56:08.823680 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:08.822830    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4" event={"ID":"6384f50d-3bbe-47ef-8e56-5a72d8113c6f","Type":"ContainerStarted","Data":"9780fb1d56c0397763b010f610fc1aa2b0a413e0df184563c15d73117ac9ed9d"}
Nov 02 11:56:08.825837 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:08.825632    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/0.log"
Nov 02 11:56:08.825837 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:08.825675    2580 generic.go:356] "Generic (PLEG): container finished" podID="53ea0d03-1f13-4ee1-af04-9ddf449d6b33" containerID="38cb200b95170077c4dd4743cd4975fd53c998b01e0cbd0347a75ad2f1cb2cd0" exitCode=255
Nov 02 11:56:08.825837 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:08.825768    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64" event={"ID":"53ea0d03-1f13-4ee1-af04-9ddf449d6b33","Type":"ContainerDied","Data":"38cb200b95170077c4dd4743cd4975fd53c998b01e0cbd0347a75ad2f1cb2cd0"}
Nov 02 11:56:08.826053 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:08.825993    2580 scope.go:122] "RemoveContainer" containerID="38cb200b95170077c4dd4743cd4975fd53c998b01e0cbd0347a75ad2f1cb2cd0"
Nov 02 11:56:08.831821 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:08.831792    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf" event={"ID":"691a1fbe-dd29-4579-8e72-8c36c2b1eda2","Type":"ContainerStarted","Data":"7978ff5a32a1fb97eaba5670abb89fa6708f6695a68dadc060145d954d13a237"}
Nov 02 11:56:09.057523 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:09.057465    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4" podStartSLOduration=367.299892268 podStartE2EDuration="6m13.057445189s" totalImagesPullingTime="5.757552921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 11:49:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:56:08.940204089 +0000 UTC m=+37.825028372" watchObservedRunningTime="2026-11-02 11:56:09.057445189 +0000 UTC m=+37.942269471"
Nov 02 11:56:09.180807 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:09.180697    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf" podStartSLOduration=368.0264117 podStartE2EDuration="6m13.180677978s" totalImagesPullingTime="5.154266278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 11:49:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:56:09.173637815 +0000 UTC m=+38.058462110" watchObservedRunningTime="2026-11-02 11:56:09.180677978 +0000 UTC m=+38.065502261"
Nov 02 11:56:09.837141 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:09.837108    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tlvw4" event={"ID":"13d85b4b-e889-48b8-97af-f23632533bea","Type":"ContainerStarted","Data":"be19378e49af974c8a16e306e66fb986380a587b49daf735d3f735446a8a6138"}
Nov 02 11:56:09.839648 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:09.839624    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/1.log"
Nov 02 11:56:09.840082 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:09.840062    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/0.log"
Nov 02 11:56:09.840173 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:09.840108    2580 generic.go:356] "Generic (PLEG): container finished" podID="53ea0d03-1f13-4ee1-af04-9ddf449d6b33" containerID="c47445652e8495da228c46a9bb8c8775699f2a05b984ce730a61e30adc8d8cfd" exitCode=255
Nov 02 11:56:09.840223 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:09.840194    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64" event={"ID":"53ea0d03-1f13-4ee1-af04-9ddf449d6b33","Type":"ContainerDied","Data":"c47445652e8495da228c46a9bb8c8775699f2a05b984ce730a61e30adc8d8cfd"}
Nov 02 11:56:09.840277 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:09.840249    2580 scope.go:122] "RemoveContainer" containerID="38cb200b95170077c4dd4743cd4975fd53c998b01e0cbd0347a75ad2f1cb2cd0"
Nov 02 11:56:09.840736 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:09.840456    2580 scope.go:122] "RemoveContainer" containerID="c47445652e8495da228c46a9bb8c8775699f2a05b984ce730a61e30adc8d8cfd"
Nov 02 11:56:09.840736 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:09.840680    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2qq64_openshift-console-operator(53ea0d03-1f13-4ee1-af04-9ddf449d6b33)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64" podUID="53ea0d03-1f13-4ee1-af04-9ddf449d6b33"
Nov 02 11:56:09.842674 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:09.842615    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-rhl9j" event={"ID":"9176bf1e-471e-43a2-af6b-650c5a494efa","Type":"ContainerStarted","Data":"145d3b348fa0332337a96851d2937b4a9b2550767b90a22c53c352510130596b"}
Nov 02 11:56:09.952098 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:09.952065    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:09.952264 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:09.952109    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:09.952264 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:09.952163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:09.952264 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:09.952198    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:09.952264 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:09.952233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:09.952264 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:09.952261    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:17.952237127 +0000 UTC m=+46.837061405 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : configmap references non-existent config key: service-ca.crt
Nov 02 11:56:09.952516 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:09.952293    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:09.952516 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:09.952325    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 11:56:09.952516 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:09.952342    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 11:56:09.952516 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:09.952369    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 11:56:09.952516 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:09.952400    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75766bbfd6-gpgxx: secret "image-registry-tls" not found
Nov 02 11:56:09.952516 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:09.952404    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 11:56:09.952516 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:09.952347    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:09.952516 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:09.952381    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert podName:c0e3ecc0-e47b-4612-87aa-53ce8434f5a8 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:17.952368859 +0000 UTC m=+46.837193123 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-hflkz" (UID: "c0e3ecc0-e47b-4612-87aa-53ce8434f5a8") : secret "networking-console-plugin-cert" not found
Nov 02 11:56:09.952516 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:09.952437    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 11:56:09.952516 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:09.952455    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 11:56:09.952516 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:09.952464    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:17.952447979 +0000 UTC m=+46.837272242 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : secret "router-metrics-certs-default" not found
Nov 02 11:56:09.952516 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:09.952515    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls podName:2759fb5a-17ad-4c29-b9c8-175769f53069 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:17.952500865 +0000 UTC m=+46.837325125 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls") pod "image-registry-75766bbfd6-gpgxx" (UID: "2759fb5a-17ad-4c29-b9c8-175769f53069") : secret "image-registry-tls" not found
Nov 02 11:56:09.953077 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:09.952536    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls podName:81050700-fe2d-45db-b66e-c83400cb2d35 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:17.952525846 +0000 UTC m=+46.837350108 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2dm77" (UID: "81050700-fe2d-45db-b66e-c83400cb2d35") : secret "cluster-monitoring-operator-tls" not found
Nov 02 11:56:09.953077 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:09.952554    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls podName:bbfab1b5-14db-477a-8f06-7cf2c1d058a2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:17.952548608 +0000 UTC m=+46.837372867 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls") pod "dns-default-cttl7" (UID: "bbfab1b5-14db-477a-8f06-7cf2c1d058a2") : secret "dns-default-metrics-tls" not found
Nov 02 11:56:09.953077 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:09.952574    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls podName:f38c9faa-5fae-45ba-93f6-e252b6f092fd nodeName:}" failed. No retries permitted until 2026-11-02 11:56:17.952564908 +0000 UTC m=+46.837389167 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-9vksm" (UID: "f38c9faa-5fae-45ba-93f6-e252b6f092fd") : secret "samples-operator-tls" not found
Nov 02 11:56:09.953077 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:09.952636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:09.953077 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:09.952724    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 11:56:09.953077 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:09.952734    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d58f4b759-ts6c2: secret "image-registry-tls" not found
Nov 02 11:56:09.953077 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:09.952765    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls podName:e9dc2cae-7e99-4282-a02d-51fb0f12b43b nodeName:}" failed. No retries permitted until 2026-11-02 11:56:17.952755063 +0000 UTC m=+46.837579364 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls") pod "image-registry-6d58f4b759-ts6c2" (UID: "e9dc2cae-7e99-4282-a02d-51fb0f12b43b") : secret "image-registry-tls" not found
Nov 02 11:56:10.072969 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.072899    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-rhl9j" podStartSLOduration=368.606389207 podStartE2EDuration="6m14.072882443s" totalImagesPullingTime="5.466493236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 11:49:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:56:10.071913086 +0000 UTC m=+38.956737368" watchObservedRunningTime="2026-11-02 11:56:10.072882443 +0000 UTC m=+38.957706723"
Nov 02 11:56:10.073550 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.073504    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tlvw4" podStartSLOduration=2.832244288 podStartE2EDuration="9.073492407s" totalImagesPullingTime="6.241248119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 11:56:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:56:09.954176944 +0000 UTC m=+38.839001225" watchObservedRunningTime="2026-11-02 11:56:10.073492407 +0000 UTC m=+38.958316689"
Nov 02 11:56:10.169744 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.168245    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-krn29"]
Nov 02 11:56:10.182513 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.182489    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:10.235785 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.235758    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-krn29"]
Nov 02 11:56:10.255835 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.255807    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/252df0cc-a5c0-4dea-aca5-bd6b08262d38-crio-socket\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:10.255974 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.255885    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:10.255974 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.255905    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lj85s\" (UniqueName: \"kubernetes.io/projected/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-api-access-lj85s\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:10.256062 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.255976    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/252df0cc-a5c0-4dea-aca5-bd6b08262d38-data-volume\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:10.256062 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.255994    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:10.281102 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.281073    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-gdvmz"]
Nov 02 11:56:10.289597 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.289568    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 02 11:56:10.289844 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.289826    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4wp8v\""
Nov 02 11:56:10.290626 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.290609    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 02 11:56:10.298657 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.298632    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gdvmz"
Nov 02 11:56:10.342224 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.342150    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-gdvmz"]
Nov 02 11:56:10.356706 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.356677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/252df0cc-a5c0-4dea-aca5-bd6b08262d38-crio-socket\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:10.356857 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.356820    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/252df0cc-a5c0-4dea-aca5-bd6b08262d38-crio-socket\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:10.356916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.356873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:10.356916 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.356904    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj85s\" (UniqueName: \"kubernetes.io/projected/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-api-access-lj85s\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:10.357101 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:10.356949    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 11:56:10.357101 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.356972    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lh5c\" (UniqueName: \"kubernetes.io/projected/6238f603-d9da-4f8a-98a8-31e69a11b625-kube-api-access-4lh5c\") pod \"migrator-66df5d496b-gdvmz\" (UID: \"6238f603-d9da-4f8a-98a8-31e69a11b625\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gdvmz"
Nov 02 11:56:10.357101 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.356991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/252df0cc-a5c0-4dea-aca5-bd6b08262d38-data-volume\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:10.357101 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:10.357011    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls podName:252df0cc-a5c0-4dea-aca5-bd6b08262d38 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:10.856991255 +0000 UTC m=+39.741815529 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-krn29" (UID: "252df0cc-a5c0-4dea-aca5-bd6b08262d38") : secret "insights-runtime-extractor-tls" not found
Nov 02 11:56:10.357101 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.357092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:10.357343 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:10.357206    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 11:56:10.357343 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:10.357261    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-rbac-proxy-cm podName:252df0cc-a5c0-4dea-aca5-bd6b08262d38 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:10.857244781 +0000 UTC m=+39.742069054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-rbac-proxy-cm") pod "insights-runtime-extractor-krn29" (UID: "252df0cc-a5c0-4dea-aca5-bd6b08262d38") : configmap "kube-rbac-proxy" not found
Nov 02 11:56:10.357416 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.357344    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/252df0cc-a5c0-4dea-aca5-bd6b08262d38-data-volume\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:10.389546 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.389486    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-n2k42\""
Nov 02 11:56:10.389546 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.389502    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 02 11:56:10.389546 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.389487    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 02 11:56:10.422534 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.422498    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj85s\" (UniqueName: \"kubernetes.io/projected/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-api-access-lj85s\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:10.457685 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.457657    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lh5c\" (UniqueName: \"kubernetes.io/projected/6238f603-d9da-4f8a-98a8-31e69a11b625-kube-api-access-4lh5c\") pod \"migrator-66df5d496b-gdvmz\" (UID: \"6238f603-d9da-4f8a-98a8-31e69a11b625\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gdvmz"
Nov 02 11:56:10.518734 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.518699    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lh5c\" (UniqueName: \"kubernetes.io/projected/6238f603-d9da-4f8a-98a8-31e69a11b625-kube-api-access-4lh5c\") pod \"migrator-66df5d496b-gdvmz\" (UID: \"6238f603-d9da-4f8a-98a8-31e69a11b625\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gdvmz"
Nov 02 11:56:10.626010 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.625977    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gdvmz"
Nov 02 11:56:10.852118 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.852074    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2vw9j" event={"ID":"cb734498-5d2a-40c1-9d1f-443e3e8c8f87","Type":"ContainerStarted","Data":"13a7b0d4944af5e9e88a284f28706429dfbd1d563757cb713efb61c4dc16e1d6"}
Nov 02 11:56:10.852588 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.852171    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:56:10.853721 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.853700    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-g7gnd" event={"ID":"98301ff8-eb60-43d4-857f-da7d8da3858b","Type":"ContainerStarted","Data":"0f7b05cadb64f8e1747a286ccc39c0ec7f69a6cc00d90c04f27ac24bbdbae088"}
Nov 02 11:56:10.855215 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.855198    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/1.log"
Nov 02 11:56:10.855553 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.855528    2580 scope.go:122] "RemoveContainer" containerID="c47445652e8495da228c46a9bb8c8775699f2a05b984ce730a61e30adc8d8cfd"
Nov 02 11:56:10.855692 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:10.855674    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2qq64_openshift-console-operator(53ea0d03-1f13-4ee1-af04-9ddf449d6b33)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64" podUID="53ea0d03-1f13-4ee1-af04-9ddf449d6b33"
Nov 02 11:56:10.862015 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.861991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:10.862104 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.862088    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:10.862156 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:10.862134    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 11:56:10.862210 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:10.862190    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 11:56:10.862250 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:10.862210    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls podName:252df0cc-a5c0-4dea-aca5-bd6b08262d38 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:11.862191866 +0000 UTC m=+40.747016140 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-krn29" (UID: "252df0cc-a5c0-4dea-aca5-bd6b08262d38") : secret "insights-runtime-extractor-tls" not found
Nov 02 11:56:10.862250 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:10.862241    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-rbac-proxy-cm podName:252df0cc-a5c0-4dea-aca5-bd6b08262d38 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:11.862230018 +0000 UTC m=+40.747054281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-rbac-proxy-cm") pod "insights-runtime-extractor-krn29" (UID: "252df0cc-a5c0-4dea-aca5-bd6b08262d38") : configmap "kube-rbac-proxy" not found
Nov 02 11:56:10.967316 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:10.967214    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2vw9j" podStartSLOduration=35.12159115 podStartE2EDuration="38.967201469s" totalImagesPullingTime="3.845610319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 11:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:56:10.966385007 +0000 UTC m=+39.851209320" watchObservedRunningTime="2026-11-02 11:56:10.967201469 +0000 UTC m=+39.852025747"
Nov 02 11:56:11.078807 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:11.078759    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-gdvmz"]
Nov 02 11:56:11.081804 ip-10-0-135-24 kubenswrapper[2580]: W1102 11:56:11.081776    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6238f603_d9da_4f8a_98a8_31e69a11b625.slice/crio-3794f898cb5401b88d8003f04d64bf4d76af411e8ed96d8cb79881a6bf27ce4f WatchSource:0}: Error finding container 3794f898cb5401b88d8003f04d64bf4d76af411e8ed96d8cb79881a6bf27ce4f: Status 404 returned error can't find the container with id 3794f898cb5401b88d8003f04d64bf4d76af411e8ed96d8cb79881a6bf27ce4f
Nov 02 11:56:11.200078 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:11.200023    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-g7gnd" podStartSLOduration=361.311303272 podStartE2EDuration="6m8.200005346s" totalImagesPullingTime="6.888702074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 11:50:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:56:11.199713825 +0000 UTC m=+40.084538099" watchObservedRunningTime="2026-11-02 11:56:11.200005346 +0000 UTC m=+40.084829630"
Nov 02 11:56:11.859959 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:11.859920    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gdvmz" event={"ID":"6238f603-d9da-4f8a-98a8-31e69a11b625","Type":"ContainerStarted","Data":"3794f898cb5401b88d8003f04d64bf4d76af411e8ed96d8cb79881a6bf27ce4f"}
Nov 02 11:56:11.871649 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:11.871612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:11.871780 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:11.871747    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 11:56:11.871838 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:11.871817    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls podName:252df0cc-a5c0-4dea-aca5-bd6b08262d38 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:13.871802826 +0000 UTC m=+42.756627084 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-krn29" (UID: "252df0cc-a5c0-4dea-aca5-bd6b08262d38") : secret "insights-runtime-extractor-tls" not found
Nov 02 11:56:11.871905 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:11.871844    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:11.871959 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:11.871918    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 02 11:56:11.871959 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:11.871958    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-rbac-proxy-cm podName:252df0cc-a5c0-4dea-aca5-bd6b08262d38 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:13.871950032 +0000 UTC m=+42.756774291 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-rbac-proxy-cm") pod "insights-runtime-extractor-krn29" (UID: "252df0cc-a5c0-4dea-aca5-bd6b08262d38") : configmap "kube-rbac-proxy" not found
Nov 02 11:56:12.461400 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:12.461362    2580 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:12.461594 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:12.461438    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:12.462002 ip-10-0-135-24 kubenswrapper[2580]: I1102 11:56:12.461974    2580 scope.go:122] "RemoveContainer" containerID="c47445652e8495da228c46a9bb8c8775699f2a05b984ce730a61e30adc8d8cfd"
Nov 02 11:56:12.462244 ip-10-0-135-24 kubenswrapper[2580]: E1102 11:56:12.462214    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2qq64_openshift-console-operator(53ea0d03-1f13-4ee1-af04-9ddf449d6b33)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64" podUID="53ea0d03-1f13-4ee1-af04-9ddf449d6b33"
Nov 02 11:56:13.352424 ip-10-0-135-24 systemd[1]: Stopping Kubernetes Kubelet...
Nov 02 11:56:13.367647 ip-10-0-135-24 systemd[1]: kubelet.service: Deactivated successfully.
Nov 02 11:56:13.367816 ip-10-0-135-24 systemd[1]: Stopped Kubernetes Kubelet.
Nov 02 11:56:13.367868 ip-10-0-135-24 systemd[1]: kubelet.service: Consumed 2.930s CPU time.
Nov 02 11:56:13.380981 ip-10-0-135-24 systemd[1]: Starting Kubernetes Kubelet...
Nov 02 11:56:13.458271 ip-10-0-135-24 kubenswrapper[4942]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 11:56:13.458271 ip-10-0-135-24 kubenswrapper[4942]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 02 11:56:13.458271 ip-10-0-135-24 kubenswrapper[4942]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 11:56:13.458271 ip-10-0-135-24 kubenswrapper[4942]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 02 11:56:13.460355 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460260    4942 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 11:56:13.460355 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460355    4942 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 11:56:13.460355 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460359    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460363    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460367    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460370    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460376    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460379    4942 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460381    4942 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460384    4942 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460387    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460390    4942 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460392    4942 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460395    4942 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460398    4942 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460400    4942 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460403    4942 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460405    4942 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460408    4942 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460411    4942 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460413    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 11:56:13.460457 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460417    4942 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460419    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460424    4942 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460428    4942 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460431    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460434    4942 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460437    4942 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460439    4942 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460442    4942 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460444    4942 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460447    4942 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460450    4942 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460452    4942 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460455    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460458    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460461    4942 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460464    4942 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460466    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460469    4942 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 11:56:13.460939 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460472    4942 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460475    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460478    4942 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460480    4942 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460483    4942 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460485    4942 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460488    4942 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460491    4942 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460493    4942 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460496    4942 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460498    4942 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460501    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460504    4942 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460507    4942 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460509    4942 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460512    4942 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460515    4942 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460518    4942 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460521    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460524    4942 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 11:56:13.461441 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460528    4942 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460532    4942 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460535    4942 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460538    4942 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460541    4942 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460544    4942 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460546    4942 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460549    4942 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460551    4942 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460554    4942 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460557    4942 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460560    4942 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460563    4942 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460565    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460568    4942 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460570    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460573    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460575    4942 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460578    4942 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460581    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 11:56:13.461929 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460583    4942 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460586    4942 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460588    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460591    4942 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460593    4942 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460596    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460599    4942 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460602    4942 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460605    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460607    4942 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460610    4942 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460613    4942 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460616    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460618    4942 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460621    4942 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460624    4942 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460627    4942 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460630    4942 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460632    4942 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 11:56:13.462463 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460635    4942 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460637    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460640    4942 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460644    4942 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460646    4942 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460649    4942 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.460652    4942 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461423    4942 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461430    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461433    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461437    4942 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461440    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461442    4942 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461446    4942 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461449    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461452    4942 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461455    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461457    4942 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461460    4942 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 11:56:13.462975 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461463    4942 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461465    4942 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461468    4942 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461471    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461474    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461477    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461480    4942 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461482    4942 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461485    4942 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461488    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461490    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461493    4942 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461495    4942 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461498    4942 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461500    4942 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461503    4942 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461505    4942 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461508    4942 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461511    4942 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461513    4942 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 11:56:13.463470 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461516    4942 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461519    4942 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461522    4942 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461524    4942 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461527    4942 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461530    4942 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461533    4942 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461535    4942 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461538    4942 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461541    4942 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461544    4942 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461547    4942 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461550    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461552    4942 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461555    4942 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461557    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461559    4942 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461562    4942 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461566    4942 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461568    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 11:56:13.463989 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461571    4942 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461573    4942 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461576    4942 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461579    4942 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461581    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461584    4942 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461586    4942 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461589    4942 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461591    4942 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461594    4942 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461596    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461599    4942 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461601    4942 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461604    4942 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461607    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461610    4942 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461612    4942 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461615    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461617    4942 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461620    4942 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 11:56:13.464516 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461623    4942 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461625    4942 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461628    4942 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461631    4942 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461633    4942 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461636    4942 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461639    4942 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461643    4942 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461646    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461649    4942 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461652    4942 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461655    4942 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461657    4942 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461660    4942 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461662    4942 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461666    4942 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461668    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461671    4942 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461673    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 11:56:13.465366 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461676    4942 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461678    4942 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461681    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461683    4942 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461686    4942 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461688    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461691    4942 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461694    4942 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461696    4942 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461701    4942 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461705    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461709    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461712    4942 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461715    4942 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.461718    4942 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462199    4942 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462204    4942 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462207    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462210    4942 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 11:56:13.465947 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462213    4942 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 11:56:13.466485 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462216    4942 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 11:56:13.466485 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462219    4942 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 11:56:13.466485 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462221    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 11:56:13.466485 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462224    4942 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 11:56:13.466485 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462228    4942 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 11:56:13.466485 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462232    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 11:56:13.466485 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462236    4942 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 11:56:13.466485 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462239    4942 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 11:56:13.466485 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462241    4942 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 11:56:13.466485 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462244    4942 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 11:56:13.466485 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462247    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 11:56:13.466485 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462250    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 11:56:13.466485 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462254    4942 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 11:56:13.466485 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462257    4942 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 11:56:13.466485 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462260    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 11:56:13.466485 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462263    4942 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 11:56:13.466485 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462268    4942 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 11:56:13.466485 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462271    4942 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462273    4942 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462277    4942 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462279    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462282    4942 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462285    4942 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462287    4942 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462290    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462292    4942 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462308    4942 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462311    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462314    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462316    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462319    4942 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462322    4942 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462325    4942 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462328    4942 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462331    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462334    4942 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462337    4942 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 11:56:13.466933 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462339    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462342    4942 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462344    4942 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462347    4942 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462350    4942 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462352    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462355    4942 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462357    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462360    4942 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462362    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462365    4942 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462369    4942 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462373    4942 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462376    4942 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462378    4942 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462381    4942 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462384    4942 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462387    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462390    4942 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462392    4942 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 11:56:13.467450 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462395    4942 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462397    4942 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462400    4942 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462403    4942 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462405    4942 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462408    4942 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462411    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462413    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462416    4942 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462418    4942 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462421    4942 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462423    4942 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462426    4942 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462428    4942 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462431    4942 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462434    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462436    4942 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462439    4942 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462441    4942 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 11:56:13.468207 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462444    4942 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462446    4942 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462449    4942 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462451    4942 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462457    4942 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462461    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462464    4942 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462467    4942 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462469    4942 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462472    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462476    4942 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462478    4942 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462481    4942 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462484    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462487    4942 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462489    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462492    4942 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462494    4942 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462497    4942 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462499    4942 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 11:56:13.468716 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462502    4942 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462505    4942 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462507    4942 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462510    4942 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.462512    4942 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462600    4942 flags.go:64] FLAG: --address="0.0.0.0"
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462607    4942 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462613    4942 flags.go:64] FLAG: --anonymous-auth="true"
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462618    4942 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462623    4942 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462626    4942 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462632    4942 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462636    4942 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462640    4942 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462643    4942 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462646    4942 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462651    4942 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462655    4942 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462658    4942 flags.go:64] FLAG: --cgroup-root=""
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462661    4942 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462664    4942 flags.go:64] FLAG: --client-ca-file=""
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462667    4942 flags.go:64] FLAG: --cloud-provider="external"
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462671    4942 flags.go:64] FLAG: --cluster-dns="[]"
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462675    4942 flags.go:64] FLAG: --cluster-domain=""
Nov 02 11:56:13.469218 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462678    4942 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462681    4942 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462685    4942 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462688    4942 flags.go:64] FLAG: --container-log-max-files="5"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462692    4942 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462695    4942 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462698    4942 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462701    4942 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462704    4942 flags.go:64] FLAG: --contention-profiling="false"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462707    4942 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462710    4942 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462713    4942 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462716    4942 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462720    4942 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462723    4942 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462726    4942 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462729    4942 flags.go:64] FLAG: --enable-load-reader="false"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462732    4942 flags.go:64] FLAG: --enable-server="true"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462736    4942 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462740    4942 flags.go:64] FLAG: --event-burst="100"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462743    4942 flags.go:64] FLAG: --event-qps="50"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462746    4942 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462752    4942 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462755    4942 flags.go:64] FLAG: --eviction-hard=""
Nov 02 11:56:13.469826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462761    4942 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462764    4942 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462767    4942 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462770    4942 flags.go:64] FLAG: --eviction-soft=""
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462773    4942 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462776    4942 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462779    4942 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462782    4942 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462785    4942 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462788    4942 flags.go:64] FLAG: --fail-swap-on="true"
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462791    4942 flags.go:64] FLAG: --feature-gates=""
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462795    4942 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462798    4942 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462801    4942 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462804    4942 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462807    4942 flags.go:64] FLAG: --healthz-port="10248"
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462810    4942 flags.go:64] FLAG: --help="false"
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462814    4942 flags.go:64] FLAG: --hostname-override="ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462817    4942 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462820    4942 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462823    4942 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462827    4942 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462830    4942 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 02 11:56:13.470456 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462833    4942 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462836    4942 flags.go:64] FLAG: --image-service-endpoint=""
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462839    4942 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462842    4942 flags.go:64] FLAG: --kube-api-burst="100"
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462845    4942 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462848    4942 flags.go:64] FLAG: --kube-api-qps="50"
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462851    4942 flags.go:64] FLAG: --kube-reserved=""
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462854    4942 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462857    4942 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462861    4942 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462864    4942 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462867    4942 flags.go:64] FLAG: --lock-file=""
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462869    4942 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462872    4942 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462875    4942 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462881    4942 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462884    4942 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462887    4942 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462890    4942 flags.go:64] FLAG: --logging-format="text"
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462893    4942 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462896    4942 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462899    4942 flags.go:64] FLAG: --manifest-url=""
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462902    4942 flags.go:64] FLAG: --manifest-url-header=""
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462907    4942 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462910    4942 flags.go:64] FLAG: --max-open-files="1000000"
Nov 02 11:56:13.471027 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462914    4942 flags.go:64] FLAG: --max-pods="110"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462916    4942 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462920    4942 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462923    4942 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462926    4942 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462929    4942 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462931    4942 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462934    4942 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462941    4942 flags.go:64] FLAG: --node-status-max-images="50"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462946    4942 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462949    4942 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462952    4942 flags.go:64] FLAG: --pod-cidr=""
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462955    4942 flags.go:64] FLAG: --pod-manifest-path=""
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462958    4942 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462961    4942 flags.go:64] FLAG: --pods-per-core="0"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462964    4942 flags.go:64] FLAG: --port="10250"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462969    4942 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462972    4942 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-03b0be0e6fd76ab82"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462975    4942 flags.go:64] FLAG: --qos-reserved=""
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462978    4942 flags.go:64] FLAG: --read-only-port="10255"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462982    4942 flags.go:64] FLAG: --register-node="true"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462985    4942 flags.go:64] FLAG: --register-with-taints=""
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462989    4942 flags.go:64] FLAG: --registry-burst="10"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462992    4942 flags.go:64] FLAG: --registry-qps="5"
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462995    4942 flags.go:64] FLAG: --reserved-cpus=""
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.462998    4942 flags.go:64] FLAG: --reserved-memory=""
Nov 02 11:56:13.471684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463002    4942 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463005    4942 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463008    4942 flags.go:64] FLAG: --rotate-certificates="false"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463011    4942 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463014    4942 flags.go:64] FLAG: --runonce="false"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463017    4942 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463020    4942 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463023    4942 flags.go:64] FLAG: --seccomp-default="false"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463026    4942 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463029    4942 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463032    4942 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463035    4942 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463038    4942 flags.go:64] FLAG: --storage-driver-password="root"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463041    4942 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463044    4942 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463048    4942 flags.go:64] FLAG: --storage-driver-user="root"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463051    4942 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463054    4942 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463057    4942 flags.go:64] FLAG: --system-cgroups=""
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463061    4942 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463066    4942 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463069    4942 flags.go:64] FLAG: --tls-cert-file=""
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463073    4942 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463077    4942 flags.go:64] FLAG: --tls-min-version=""
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463080    4942 flags.go:64] FLAG: --tls-private-key-file=""
Nov 02 11:56:13.472431 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463083    4942 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463086    4942 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463089    4942 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463092    4942 flags.go:64] FLAG: --v="2"
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463096    4942 flags.go:64] FLAG: --version="false"
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463100    4942 flags.go:64] FLAG: --vmodule=""
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463104    4942 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463108    4942 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463228    4942 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463232    4942 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463235    4942 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463238    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463241    4942 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463243    4942 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463247    4942 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463252    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463255    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463257    4942 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463260    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463263    4942 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463266    4942 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 11:56:13.473094 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463268    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463273    4942 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463276    4942 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463279    4942 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463281    4942 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463284    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463286    4942 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463289    4942 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463293    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463308    4942 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463311    4942 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463314    4942 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463316    4942 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463319    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463322    4942 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463325    4942 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463327    4942 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463330    4942 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463332    4942 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463335    4942 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 11:56:13.473814 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463338    4942 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463341    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463343    4942 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463346    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463348    4942 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463351    4942 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463353    4942 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463356    4942 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463359    4942 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463361    4942 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463364    4942 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463367    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463369    4942 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463374    4942 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463377    4942 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463379    4942 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463382    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463384    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463387    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463391    4942 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 11:56:13.474750 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463395    4942 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463398    4942 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463401    4942 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463404    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463407    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463409    4942 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463412    4942 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463414    4942 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463417    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463419    4942 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463422    4942 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463424    4942 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463427    4942 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463430    4942 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463432    4942 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463435    4942 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463438    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463440    4942 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463443    4942 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 11:56:13.475547 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463446    4942 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463448    4942 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463451    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463453    4942 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463456    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463458    4942 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463462    4942 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463465    4942 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463467    4942 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463470    4942 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463473    4942 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463476    4942 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463478    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463482    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463485    4942 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463488    4942 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463490    4942 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463493    4942 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463495    4942 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463498    4942 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 11:56:13.476877 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463500    4942 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 11:56:13.477496 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463503    4942 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 11:56:13.477496 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463505    4942 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 11:56:13.477496 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463508    4942 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 11:56:13.477496 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463511    4942 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 11:56:13.477496 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463513    4942 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 11:56:13.477496 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463516    4942 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 11:56:13.477496 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463518    4942 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 11:56:13.477496 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463521    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 11:56:13.477496 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463524    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 11:56:13.477496 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463526    4942 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 11:56:13.477496 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463528    4942 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 11:56:13.477496 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463531    4942 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 11:56:13.477496 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463533    4942 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 11:56:13.477496 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.463558    4942 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 11:56:13.477496 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463591    4942 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463595    4942 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463598    4942 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463602    4942 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463604    4942 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463607    4942 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463610    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463612    4942 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463615    4942 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463617    4942 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463620    4942 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463622    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463625    4942 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463627    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463630    4942 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463632    4942 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463635    4942 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463638    4942 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463640    4942 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463643    4942 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 11:56:13.478106 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463646    4942 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463648    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463651    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463653    4942 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463656    4942 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463659    4942 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463661    4942 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463664    4942 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463666    4942 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463669    4942 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463671    4942 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463674    4942 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463676    4942 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463679    4942 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463681    4942 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463685    4942 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463688    4942 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463690    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463693    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 11:56:13.478752 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463696    4942 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463698    4942 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463701    4942 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463703    4942 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463706    4942 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463708    4942 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463711    4942 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463714    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463716    4942 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463719    4942 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463723    4942 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463726    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463730    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463733    4942 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463736    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463738    4942 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463741    4942 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463743    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463746    4942 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 11:56:13.479251 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463749    4942 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463752    4942 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463754    4942 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463757    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463759    4942 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463762    4942 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463764    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463767    4942 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463769    4942 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463773    4942 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463775    4942 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463778    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463781    4942 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463783    4942 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463786    4942 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463788    4942 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463791    4942 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463793    4942 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463796    4942 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463798    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 11:56:13.479800 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463801    4942 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463803    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463806    4942 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463808    4942 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463811    4942 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463813    4942 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463815    4942 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463818    4942 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463820    4942 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463823    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463825    4942 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463828    4942 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463830    4942 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463833    4942 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463835    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463838    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463840    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463844    4942 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463847    4942 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463852    4942 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 11:56:13.480354 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463856    4942 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463860    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463863    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463865    4942 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463868    4942 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463871    4942 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463873    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.463876    4942 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.467895    4942 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.467912    4942 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.467929    4942 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.467934    4942 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.467992    4942 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.467997    4942 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468000    4942 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468003    4942 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468006    4942 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468009    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468012    4942 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 11:56:13.480994 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468015    4942 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468017    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468020    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468023    4942 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468025    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468028    4942 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468031    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468033    4942 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468036    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468039    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468042    4942 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468045    4942 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468048    4942 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468050    4942 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468053    4942 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468056    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468059    4942 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468061    4942 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468064    4942 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468067    4942 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 11:56:13.481644 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468070    4942 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468076    4942 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468079    4942 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468082    4942 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468084    4942 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468087    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468090    4942 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468093    4942 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468095    4942 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468098    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468101    4942 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468103    4942 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468106    4942 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468109    4942 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468112    4942 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468115    4942 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468117    4942 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468120    4942 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468123    4942 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468126    4942 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 11:56:13.482223 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468129    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468132    4942 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468134    4942 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468137    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468140    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468142    4942 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468145    4942 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468147    4942 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468152    4942 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468156    4942 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468159    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468163    4942 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468166    4942 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468169    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468172    4942 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468174    4942 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468177    4942 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468179    4942 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468182    4942 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 11:56:13.482789 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468185    4942 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468188    4942 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468190    4942 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468193    4942 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468195    4942 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468198    4942 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468201    4942 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468203    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468206    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468209    4942 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468211    4942 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468214    4942 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468217    4942 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468220    4942 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468223    4942 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468226    4942 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468229    4942 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468231    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468233    4942 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468236    4942 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 11:56:13.483249 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468239    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468241    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468244    4942 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468246    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468249    4942 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468252    4942 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468254    4942 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468257    4942 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468260    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468262    4942 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468265    4942 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468267    4942 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468270    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468272    4942 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468275    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468277    4942 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468281    4942 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468285    4942 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468287    4942 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 11:56:13.483805 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468290    4942 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 11:56:13.484272 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.468337    4942 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 11:56:13.484272 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468499    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 11:56:13.484272 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468505    4942 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 11:56:13.484272 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468508    4942 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 11:56:13.484272 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468511    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 11:56:13.484272 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468514    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 11:56:13.484272 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468518    4942 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 11:56:13.484272 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468521    4942 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 11:56:13.484272 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468524    4942 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 11:56:13.484272 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468526    4942 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 11:56:13.484272 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468529    4942 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 11:56:13.484272 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468532    4942 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 11:56:13.484272 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468534    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 11:56:13.484272 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468537    4942 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 11:56:13.484272 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468540    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 11:56:13.484272 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468543    4942 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468545    4942 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468548    4942 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468550    4942 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468553    4942 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468556    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468558    4942 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468561    4942 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468563    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468566    4942 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468569    4942 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468572    4942 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468574    4942 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468577    4942 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468580    4942 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468583    4942 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468585    4942 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468588    4942 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468590    4942 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 11:56:13.484705 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468595    4942 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468598    4942 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468604    4942 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468607    4942 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468609    4942 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468613    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468616    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468618    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468621    4942 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468624    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468626    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468629    4942 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468633    4942 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468636    4942 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468639    4942 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468641    4942 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468644    4942 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468646    4942 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468649    4942 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 11:56:13.485206 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468652    4942 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468655    4942 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468658    4942 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468660    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468663    4942 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468665    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468668    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468670    4942 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468673    4942 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468676    4942 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468679    4942 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468681    4942 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468684    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468686    4942 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468689    4942 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468693    4942 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468696    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468699    4942 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468702    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468705    4942 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 11:56:13.485717 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468707    4942 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468710    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468713    4942 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468715    4942 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468718    4942 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468722    4942 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468725    4942 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468727    4942 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468730    4942 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468733    4942 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468736    4942 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468738    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468741    4942 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468743    4942 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468746    4942 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468749    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468751    4942 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468754    4942 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468757    4942 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468759    4942 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 11:56:13.486198 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468762    4942 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 11:56:13.486807 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468765    4942 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 11:56:13.486807 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468767    4942 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 11:56:13.486807 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468770    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 11:56:13.486807 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468773    4942 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 11:56:13.486807 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468775    4942 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 11:56:13.486807 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468778    4942 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 11:56:13.486807 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468788    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 11:56:13.486807 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468792    4942 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 11:56:13.486807 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468795    4942 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 11:56:13.486807 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468799    4942 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 11:56:13.486807 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468802    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 11:56:13.486807 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468806    4942 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 11:56:13.486807 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468809    4942 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 11:56:13.486807 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.468832    4942 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 02 11:56:13.486807 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468864    4942 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 02 11:56:13.486807 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468868    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468871    4942 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468876    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468879    4942 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468882    4942 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468885    4942 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468887    4942 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468890    4942 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468893    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468896    4942 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468898    4942 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468901    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468903    4942 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468906    4942 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468909    4942 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468911    4942 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468914    4942 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468929    4942 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468933    4942 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 02 11:56:13.487565 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468937    4942 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468939    4942 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468942    4942 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468946    4942 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468951    4942 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468954    4942 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468957    4942 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468960    4942 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468962    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468965    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468968    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468971    4942 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468974    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468976    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468979    4942 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468982    4942 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468985    4942 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468988    4942 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468990    4942 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 02 11:56:13.488343 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468993    4942 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468995    4942 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.468998    4942 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469000    4942 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469003    4942 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469006    4942 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469009    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469011    4942 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469014    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469016    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469019    4942 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469021    4942 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469024    4942 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469027    4942 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469030    4942 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469032    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469035    4942 feature_gate.go:386] unrecognized feature gate: Example
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469038    4942 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469040    4942 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469043    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 02 11:56:13.489146 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469045    4942 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469048    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469051    4942 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469053    4942 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469056    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469059    4942 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469062    4942 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469064    4942 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469068    4942 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469070    4942 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469073    4942 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469076    4942 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469078    4942 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469081    4942 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469084    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469087    4942 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469089    4942 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469092    4942 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469094    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469097    4942 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 02 11:56:13.489875 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469100    4942 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469102    4942 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469105    4942 feature_gate.go:386] unrecognized feature gate: Example2
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469107    4942 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469110    4942 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469113    4942 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469117    4942 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469120    4942 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469122    4942 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469125    4942 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469127    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469130    4942 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469132    4942 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469136    4942 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469140    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469143    4942 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469146    4942 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469149    4942 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469152    4942 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 02 11:56:13.490476 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469154    4942 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 02 11:56:13.491090 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469160    4942 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 02 11:56:13.491090 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469164    4942 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 02 11:56:13.491090 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469167    4942 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 02 11:56:13.491090 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469169    4942 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 02 11:56:13.491090 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469172    4942 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 02 11:56:13.491090 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469175    4942 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 02 11:56:13.491090 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:13.469177    4942 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 02 11:56:13.491090 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.469230    4942 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 02 11:56:13.491090 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.470176    4942 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 02 11:56:13.491090 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.470236    4942 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 02 11:56:13.491090 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.470435    4942 server.go:1014] "Starting client certificate rotation"
Nov 02 11:56:13.491090 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.470572    4942 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 02 11:56:13.491090 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.470623    4942 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-01 11:50:31 +0000 UTC" deadline="2028-07-26 23:05:52.109253642 +0000 UTC"
Nov 02 11:56:13.491090 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.470649    4942 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15179h9m38.638606655s"
Nov 02 11:56:13.491090 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.470891    4942 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 11:56:13.491090 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.471968    4942 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 02 11:56:13.491538 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.473253    4942 log.go:25] "Validated CRI v1 runtime API"
Nov 02 11:56:13.491538 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.475392    4942 log.go:25] "Validated CRI v1 image API"
Nov 02 11:56:13.491538 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.476161    4942 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 02 11:56:13.491538 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.478912    4942 fs.go:135] Filesystem UUIDs: map[58732fc7-915a-4274-9f43-b2de0be07202:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 7fdad5ae-d3e0-4035-acc0-4b531e2ebf33:/dev/nvme0n1p4]
Nov 02 11:56:13.491676 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.478929    4942 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/157531b7c6576657c1682cfa3945f3764b167240c374c1029cdb9badcf0b03e7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/157531b7c6576657c1682cfa3945f3764b167240c374c1029cdb9badcf0b03e7/userdata/shm major:0 minor:211 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1626b253692ce32927cd1c75ee25cbd3815299edb3bb10ccc24efaf9075aa336/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1626b253692ce32927cd1c75ee25cbd3815299edb3bb10ccc24efaf9075aa336/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2f3ecd16e39a81d34aa0291317a80fa6fdeeff614be213fcf4c761a24ee92b4e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2f3ecd16e39a81d34aa0291317a80fa6fdeeff614be213fcf4c761a24ee92b4e/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2fad6c8bb3e025df2fcd1778e6bc1e7ad6d715f97a65b8e02e6ca4c254806413/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2fad6c8bb3e025df2fcd1778e6bc1e7ad6d715f97a65b8e02e6ca4c254806413/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/300e91d7e2bd6fd9552f384c619772601ce569e29f65a2c676b716953c72a443/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/300e91d7e2bd6fd9552f384c619772601ce569e29f65a2c676b716953c72a443/userdata/shm major:0 minor:295 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/37670025ba6d0060a48cbaf6065cef83c2fbd01e7a266b2180386d9fa1b75731/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/37670025ba6d0060a48cbaf6065cef83c2fbd01e7a266b2180386d9fa1b75731/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3794f898cb5401b88d8003f04d64bf4d76af411e8ed96d8cb79881a6bf27ce4f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3794f898cb5401b88d8003f04d64bf4d76af411e8ed96d8cb79881a6bf27ce4f/userdata/shm major:0 minor:340 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4710cea14069f1ce66928eff09f9ff1fa2ab600549eeb4737a1075940c93d2f0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4710cea14069f1ce66928eff09f9ff1fa2ab600549eeb4737a1075940c93d2f0/userdata/shm major:0 minor:248 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4b5df228477cc031b603768652830fa1d1af537c320e2788ab2e44c2d0663118/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4b5df228477cc031b603768652830fa1d1af537c320e2788ab2e44c2d0663118/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/528c27028e7efb73e18821bef8122692542cf1cade92a3a969ea56447b01f550/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/528c27028e7efb73e18821bef8122692542cf1cade92a3a969ea56447b01f550/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7717a04a35c612622fe1587f3cf2f5d814bab7717ee54f48a2e4a319a83546d5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7717a04a35c612622fe1587f3cf2f5d814bab7717ee54f48a2e4a319a83546d5/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7ad8b843416b85c01aba0c78e0dc85ec40f2b20bc8bbc422cd2e3f5f4f0551e9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7ad8b843416b85c01aba0c78e0dc85ec40f2b20bc8bbc422cd2e3f5f4f0551e9/userdata/shm major:0 minor:77 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8d4366f1cd075adb9f1bdcdbdcff5bd0278d670a602795381004153b9933001b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8d4366f1cd075adb9f1bdcdbdcff5bd0278d670a602795381004153b9933001b/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9ce07e1cd2e465b2d044ecc7731bc1dba56885c01c9a992463bd713d620cb571/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9ce07e1cd2e465b2d044ecc7731bc1dba56885c01c9a992463bd713d620cb571/userdata/shm major:0 minor:85 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a129b2ee37d0619b409638f2e04aad33408dd2dd73f19fec71d9c40bd039ad35/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a129b2ee37d0619b409638f2e04aad33408dd2dd73f19fec71d9c40bd039ad35/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a2ccfcfa40dba291334b84b4bc17c12052b8b1d5979211e1e6778c097be27411/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a2ccfcfa40dba291334b84b4bc17c12052b8b1d5979211e1e6778c097be27411/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ba2958d5085f0550a4106550d02a1517e4f0b925a9e2a5a83b34e0bcf8baecb5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ba2958d5085f0550a4106550d02a1517e4f0b925a9e2a5a83b34e0bcf8baecb5/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c9f277d97a60b0ae286a17629a77187bc220838b3174007f5e7935e23b963dbc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c9f277d97a60b0ae286a17629a77187bc220838b3174007f5e7935e23b963dbc/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d33d00be8285b37a83e9f995f598d1de1e53c29edb532922ea2642f91786578b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d33d00be8285b37a83e9f995f598d1de1e53c29edb532922ea2642f91786578b/userdata/shm major:0 minor:290 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dbd03d2736aff17abe3e92400f910da37ac2f3947dcca36fba592dff611e7de3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dbd03d2736aff17abe3e92400f910da37ac2f3947dcca36fba592dff611e7de3/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e6f5f644265046944ccd3504362891bab45948af07c5ceb4eb229f945beb2a60/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e6f5f644265046944ccd3504362891bab45948af07c5ceb4eb229f945beb2a60/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ec3be01ca9e36f5d00ffc5c704ed09be987f8ca357bc7a3e448d221dba99b078/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ec3be01ca9e36f5d00ffc5c704ed09be987f8ca357bc7a3e448d221dba99b078/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fa8182ba90f40af301e6472b699b54e312b9b5db522fd5e2b3bf7db970775544/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fa8182ba90f40af301e6472b699b54e312b9b5db522fd5e2b3bf7db970775544/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a093470-b06b-49c5-9220-2db08ecfde86/volumes/kubernetes.io~projected/kube-api-access-lbqbh:{mountpoint:/var/lib/kubelet/pods/0a093470-b06b-49c5-9220-2db08ecfde86/volumes/kubernetes.io~projected/kube-api-access-lbqbh major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/13d85b4b-e889-48b8-97af-f23632533bea/volumes/kubernetes.io~projected/kube-api-access-7j7r5:{mountpoint:/var/lib/kubelet/pods/13d85b4b-e889-48b8-97af-f23632533bea/volumes/kubernetes.io~projected/kube-api-access-7j7r5 major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/13d85b4b-e889-48b8-97af-f23632533bea/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/13d85b4b-e889-48b8-97af-f23632533bea/volumes/kubernetes.io~secret/cert major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/15291745-62ec-45f2-85f9-373504d79de2/volumes/kubernetes.io~projected/kube-api-access-xmwmm:{mountpoint:/var/lib/kubelet/pods/15291745-62ec-45f2-85f9-373504d79de2/volumes/kubernetes.io~projected/kube-api-access-xmwmm major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/15291745-62ec-45f2-85f9-373504d79de2/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/15291745-62ec-45f2-85f9-373504d79de2/volumes/kubernetes.io~secret/default-certificate major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/15291745-62ec-45f2-85f9-373504d79de2/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/15291745-62ec-45f2-85f9-373504d79de2/volumes/kubernetes.io~secret/stats-auth major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/252df0cc-a5c0-4dea-aca5-bd6b08262d38/volumes/kubernetes.io~projected/kube-api-access-lj85s:{mountpoint:/var/lib/kubelet/pods/252df0cc-a5c0-4dea-aca5-bd6b08262d38/volumes/kubernetes.io~projected/kube-api-access-lj85s major:0 minor:334 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/269be49a-6e81-4730-960e-cb0b4a7b031b/volumes/kubernetes.io~projected/kube-api-access-hfp9f:{mountpoint:/var/lib/kubelet/pods/269be49a-6e81-4730-960e-cb0b4a7b031b/volumes/kubernetes.io~projected/kube-api-access-hfp9f major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2759fb5a-17ad-4c29-b9c8-175769f53069/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/2759fb5a-17ad-4c29-b9c8-175769f53069/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2759fb5a-17ad-4c29-b9c8-175769f53069/volumes/kubernetes.io~projected/kube-api-access-5mg2c:{mountpoint:/var/lib/kubelet/pods/2759fb5a-17ad-4c29-b9c8-175769f53069/volumes/kubernetes.io~projected/kube-api-access-5mg2c major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2759fb5a-17ad-4c29-b9c8-175769f53069/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/2759fb5a-17ad-4c29-b9c8-175769f53069/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2759fb5a-17ad-4c29-b9c8-175769f53069/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/2759fb5a-17ad-4c29-b9c8-175769f53069/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2dc02c96-a4d9-46ab-8b52-5382cf43b92c/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/2dc02c96-a4d9-46ab-8b52-5382cf43b92c/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2dc02c96-a4d9-46ab-8b52-5382cf43b92c/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/2dc02c96-a4d9-46ab-8b52-5382cf43b92c/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2dc02c96-a4d9-46ab-8b52-5382cf43b92c/volumes/kubernetes.io~projected/kube-api-access-lbnkr:{mountpoint:/var/lib/kubelet/pods/2dc02c96-a4d9-46ab-8b52-5382cf43b92c/volumes/kubernetes.io~projected/kube-api-access-lbnkr major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4d239eb0-492f-4335-857e-4edc754823e4/volumes/kubernetes.io~projected/kube-api-access-gk6fk:{mountpoint:/var/lib/kubelet/pods/4d239eb0-492f-4335-857e-4edc754823e4/volumes/kubernetes.io~projected/kube-api-access-gk6fk major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/53ea0d03-1f13-4ee1-af04-9ddf449d6b33/volumes/kubernetes.io~projected/kube-api-access-6zstf:{mountpoint:/var/lib/kubelet/pods/53ea0d03-1f13-4ee1-af04-9ddf449d6b33/volumes/kubernetes.io~projected/kube-api-access-6zstf major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/53ea0d03-1f13-4ee1-af04-9ddf449d6b33/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/53ea0d03-1f13-4ee1-af04-9ddf449d6b33/volumes/kubernetes.io~secret/serving-cert major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5ca3e4d6-0eac-4309-b049-e5f597728119/volumes/kubernetes.io~projected/kube-api-access-7fndj:{mountpoint:/var/lib/kubelet/pods/5ca3e4d6-0eac-4309-b049-e5f597728119/volumes/kubernetes.io~projected/kube-api-access-7fndj major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5e041a46-74d9-4fc1-82e7-6c83d2b3c229/volumes/kubernetes.io~projected/kube-api-access-r554w:{mountpoint:/var/lib/kubelet/pods/5e041a46-74d9-4fc1-82e7-6c83d2b3c229/volumes/kubernetes.io~projected/kube-api-access-r554w major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/60dc0a20-c792-4343-aec8-7ff8f86955e9/volumes/kubernetes.io~projected/kube-api-access-tsnt9:{mountpoint:/var/lib/kubelet/pods/60dc0a20-c792-4343-aec8-7ff8f86955e9/volumes/kubernetes.io~projected/kube-api-access-tsnt9 major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6238f603-d9da-4f8a-98a8-31e69a11b625/volumes/kubernetes.io~projected/kube-api-access-4lh5c:{mountpoint:/var/lib/kubelet/pods/6238f603-d9da-4f8a-98a8-31e69a11b625/volumes/kubernetes.io~projected/kube-api-access-4lh5c major:0 minor:339 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6384f50d-3bbe-47ef-8e56-5a72d8113c6f/volumes/kubernetes.io~projected/kube-api-access-8v4hm:{mountpoint:/var/lib/kubelet/pods/6384f50d-3bbe-47ef-8e56-5a72d8113c6f/volumes/kubernetes.io~projected/kube-api-access-8v4hm major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6384f50d-3bbe-47ef-8e56-5a72d8113c6f/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/6384f50d-3bbe-47ef-8e56-5a72d8113c6f/volumes/kubernetes.io~secret/serving-cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/68728c63-ace5-4888-a19b-2782fd2ba673/volumes/kubernetes.io~projected/kube-api-access-hb29w:{mountpoint:/var/lib/kubelet/pods/68728c63-ace5-4888-a19b-2782fd2ba673/volumes/kubernetes.io~projected/kube-api-access-hb29w major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/691a1fbe-dd29-4579-8e72-8c36c2b1eda2/volumes/kubernetes.io~projected/kube-api-access-vdzs7:{mountpoint:/var/lib/kubelet/pods/691a1fbe-dd29-4579-8e72-8c36c2b1eda2/volumes/kubernetes.io~projected/kube-api-access-vdzs7 major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/691a1fbe-dd29-4579-8e72-8c36c2b1eda2/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/691a1fbe-dd29-4579-8e72-8c36c2b1eda2/volumes/kubernetes.io~secret/serving-cert major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6b350fc9-6bb7-43c4-9b89-c5157f42ca26/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/6b350fc9-6bb7-43c4-9b89-c5157f42ca26/volumes/kubernetes.io~secret/agent-certs major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d8ba8e4-34a6-47eb-a33f-b67c52acd079/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/6d8ba8e4-34a6-47eb-a33f-b67c52acd079/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:276 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4/volumes/kubernetes.io~projected/kube-api-access-tbnkn:{mountpoint:/var/lib/kubelet/pods/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4/volumes/kubernetes.io~projected/kube-api-access-tbnkn major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/81050700-fe2d-45db-b66e-c83400cb2d35/volumes/kubernetes.io~projected/kube-api-access-xgndt:{mountpoint:/var/lib/kubelet/pods/81050700-fe2d-45db-b66e-c83400cb2d35/volumes/kubernetes.io~projected/kube-api-access-xgndt major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9176bf1e-471e-43a2-af6b-650c5a494efa/volumes/kubernetes.io~projected/kube-api-access-b54rp:{mountpoint:/var/lib/kubelet/pods/9176bf1e-471e-43a2-af6b-650c5a494efa/volumes/kubernetes.io~projected/kube-api-access-b54rp major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9176bf1e-471e-43a2-af6b-650c5a494efa/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/9176bf1e-471e-43a2-af6b-650c5a494efa/volumes/kubernetes.io~secret/serving-cert major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/98301ff8-eb60-43d4-857f-da7d8da3858b/volumes/kubernetes.io~projected/kube-api-access-2xjgq:{mountpoint:/var/lib/kubelet/pods/98301ff8-eb60-43d4-857f-da7d8da3858b/volumes/kubernetes.io~projected/kube-api-access-2xjgq major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a05c4de6-e34a-416d-9ed9-35fa6a614dcc/volumes/kubernetes.io~projected/kube-api-access-7hxmh:{mountpoint:/var/lib/kubelet/pods/a05c4de6-e34a-416d-9ed9-35fa6a614dcc/volumes/kubernetes.io~projected/kube-api-access-7hxmh major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bbfab1b5-14db-477a-8f06-7cf2c1d058a2/volumes/kubernetes.io~projected/kube-api-access-d6jj5:{mountpoint:/var/lib/kubelet/pods/bbfab1b5-14db-477a-8f06-7cf2c1d058a2/volumes/kubernetes.io~projected/kube-api-access-d6jj5 major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cb734498-5d2a-40c1-9d1f-443e3e8c8f87/volumes/kubernetes.io~projected/kube-api-access-v7hn7:{mountpoint:/var/lib/kubelet/pods/cb734498-5d2a-40c1-9d1f-443e3e8c8f87/volumes/kubernetes.io~projected/kube-api-access-v7hn7 major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cc29f105-14d5-4004-afe0-1d0c8c17a5fb/volumes/kubernetes.io~projected/kube-api-access-nwnz6:{mountpoint:/var/lib/kubelet/pods/cc29f105-14d5-4004-afe0-1d0c8c17a5fb/volumes/kubernetes.io~projected/kube-api-access-nwnz6 major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e81ecb7e-0219-4976-b881-8c87ab61db50/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/e81ecb7e-0219-4976-b881-8c87ab61db50/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e81ecb7e-0219-4976-b881-8c87ab61db50/volumes/kubernetes.io~projected/kube-api-access-7g8ls:{mountpoint:/var/lib/kubelet/pods/e81ecb7e-0219-4976-b881-8c87ab61db50/volumes/kubernetes.io~projected/kube-api-access-7g8ls major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e81ecb7e-0219-4976-b881-8c87ab61db50/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/e81ecb7e-0219-4976-b881-8c87ab61db50/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e9dc2cae-7e99-4282-a02d-51fb0f12b43b/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/e9dc2cae-7e99-4282-a02d-51fb0f12b43b/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e9dc2cae-7e99-4282-a02d-51fb0f12b43b/volumes/kubernetes.io~projected/kube-api-access-vqzcg:{mountpoint:/var/lib/kubelet/pods/e9dc2cae-7e99-4282-a02d-51fb0f12b43b/volumes/kubernetes.io~projected/kube-api-access-vqzcg major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e9dc2cae-7e99-4282-a02d-51fb0f12b43b/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/e9dc2cae-7e99-4282-a02d-51fb0f12b43b/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e9dc2cae-7e99-4282-a02d-51fb0f12b43b/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/e9dc2cae-7e99-4282-a02d-51fb0f12b43b/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f38c9faa-5fae-45ba-93f6-e252b6f092fd/volumes/kubernetes.io~projected/kube-api-access-g9nlh:{mountpoint:/var/lib/kubelet/pods/f38c9faa-5fae-45ba-93f6-e252b6f092fd/volumes/kubernetes.io~projected/kube-api-access-g9nlh major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc644b68-76cd-4326-8a15-2f8e3dd393d0/volumes/kubernetes.io~projected/kube-api-access-tvpj5:{mountpoint:/var/lib/kubelet/pods/fc644b68-76cd-4326-8a15-2f8e3dd393d0/volumes/kubernetes.io~projected/kube-api-access-tvpj5 major:0 minor:67 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/2e654f5bc37c1f3f2db5b1c31e8e7d4e329062b87dad329655da8cf2cc12f39a/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/03ca447cd5d05528836c55c1e9080226466f8361c1eb472a3c2a9d2b730113aa/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/a5fab6b9daf4159bd3c892fa457269548d3509be669be8210366a038b9b39e5c/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/d9d11db5f232a157e5dfa0c30ffd22193ec067097de46dc37092980b6882c901/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/09d815224b9733886e5b883f59c26f9b1c4de707ad783ed2a40a06c20b1ecab3/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/8e0310bbfe41649bcd582534038ab059b61463419b2be6ba581fe4927d511f45/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/1279c01e2597ccab6f5aac4b2d887cd677e257552b649eff01921ee13e7e4377/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/34dbaf865e30930a660842bfdd99c5c035119c9494c3e647c6ec03598e084ead/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/3260d6d83afad4a9093bee267dc998dc6cfd76257c77ae342509f0506a3ba46c/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-138:{mountpoint:/var/lib/containers/storage/overlay/e6c3c23cd97c2a177f1df5fe36bc1bb6a8939b6fec2d238f83be6f9fcf8cc463/merged major:0 minor:138 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/36016dcb4fc2c43a271c44f2b990fd1498ecab5b46b41ed729d9b074a3b6bfb9/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/52b093c830068d4c74babb83de1e521a01b0fb2fec2f5b29b566e367bd492f28/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-149:{mountpoint:/var/lib/containers/storage/overlay/5ac3f0d634a5679e5c021558f56202f5a58f60a246e98c5c687a0495bdf37ed8/merged major:0 minor:149 fsType:overlay blockSize:0} overlay_0-151:{mountpoint:/var/lib/containers/storage/overlay/ddf0bacff8dfbf4a99b3498327edf5083364ebe8d0d27183dd4cff43b8e17885/merged major:0 minor:151 fsType:overlay blockSize:0} overlay_0-161:{mountpoint:/var/lib/containers/storage/overlay/65145e804d4c5034a6e56aa1f616677412efaabf95a5f74656cfcba4d2cb30cb/merged major:0 minor:161 fsType:overlay blockSize:0} overlay_0-164:{mountpoint:/var/lib/containers/storage/overlay/f36105ec495af2cf6c36982e855dd77eef30ea75c525eaa3e84288d790ec6cd4/merged major:0 minor:164 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/5b7440f654e1cbfa81b7d122285560169abab830bb63319d03d7557488d22dd0/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/ce5c08ebe0d2d96281d5ae74d5332e5e64728e4e377c37670b61947eeebb7de2/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/0c069bd6d71ff0b057b38a29bc91fcd70065baace693bfc6190ab8f0bb37a837/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/d1ca62502376d0aede8cfb89aada021608fea238257fd7c1659f4421b8b55bd3/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/f334599e040fd3f3db306555ff8728b2c667c28de20627e10be2ff922c843f96/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-246:{mountpoint:/var/lib/containers/storage/overlay/e065412bb4ec451019bd43ccacf1d05e53c6dcaf3c34c77a71a4538f91eb8580/merged major:0 minor:246 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/920034d18ce3b8f3444c70a6c9c74faa961a4a29d5b0fc55d7c2328cfafb347a/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/1dc3d3aacfb081bf5b9bfca2c67b791eb72f7b8bf526de9aec4074b47b783208/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/6064fce42347d9513345f34c8e5380d6bd00a783a964ed2549db85c57339b8f7/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/fabaa15879800873cf47240468c37ce75de6a7c4ce974feab79f6fc345b49469/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-261:{mountpoint:/var/lib/containers/storage/overlay/2ae6518d0db7cf90bba3b1a95ac3f2ce28f97c98603f3726ed0c4405dbbedf36/merged major:0 minor:261 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/66e658eeb126dada476870705e9a1706f789456d85912435b1a9f1d6427780f5/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/deaae1c22c2203e24dc70c7073f7469d63d13fdda186b7eeb98363d7238831ea/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/7f9c1b1628a4a097efb14e445a8d1899eff603fbcc246f4d557dd16985b690a8/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-270:{mountpoint:/var/lib/containers/storage/overlay/7a7db3c81379a90ea325958d6f21e3c3c8fbda8ea282fd8000ffc5b6e3527b8b/merged major:0 minor:270 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/7bf0214772fed31d6c4fc69315d816d26792f68a25eb12eb48fc9a1f7e211f44/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-274:{mountpoint:/var/lib/containers/storage/overlay/4b813a3ea822ca84de5949a473f2d6d22d056a726b77576336f52d8612f2263e/merged major:0 minor:274 fsType:overlay blockSize:0} overlay_0-281:{mountpoint:/var/lib/containers/storage/overlay/84c7d82e0c95e95be950acd04156ad58a955347f9afb12ebebcdd1d29197d589/merged major:0 minor:281 fsType:overlay blockSize:0} overlay_0-288:{mountpoint:/var/lib/containers/storage/overlay/14870d2feacca6ac8878c4e87af0f7cfb6dbb1f0b87a447c8e5f8e15c23a29e9/merged major:0 minor:288 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/883aec5b2bd76b1024e6b485943213d32a2bf2426b4d942c1633772e1d11c602/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/72bef715a7eb977ea08d2029ccea952e99d68d722b29a5373441c36113cef9c9/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/d4914428fcf14bd7e0ac826cffd8c0246f177c149a587d77d44c2fa45e88e525/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/d20732b7452f84661e425c11e698f280aa02e42e62c5f2570bf76dd578d217d6/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/0f55e66929d8bdfb6f92edd19976726c156e1e9f1b1b79f6192260977b4e3a0a/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/e363f070d80ef2313e577d2b7f8e6a70cf6897c8f2328f70f882ae4ae06db768/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-322:{mountpoint:/var/lib/containers/storage/overlay/520a0282b3914597b0d9be4e9309b2020e0d21176e47c37bb507cbc8b4f70fbe/merged major:0 minor:322 fsType:overlay blockSize:0} overlay_0-328:{mountpoint:/var/lib/containers/storage/overlay/64afd43719289f86c65e62a8329786e9cf29b432bf3526e3ef2d6aff49e845cf/merged major:0 minor:328 fsType:overlay blockSize:0} overlay_0-342:{mountpoint:/var/lib/containers/storage/overlay/5beb906bbe862b5fffa5d7f1a1f48f34c4e59131b76a44e2b966dd6196fd9124/merged major:0 minor:342 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/7b481964f2b6bf0d06a1b43985c578a501380c6a84987d4ac65946dc74bd757d/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/a88b732d45a466acc8912f66e7fe3b3d589402c2379ed1ebe066d40afb257ec5/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/7d293b64f22d4f25f0dadb8aaece63b06aae92196634140c3d92ee946120f194/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/112f0a0899067c3721e49efb115e2de9d99f197e2087cc1a7f407b06f8b035f5/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/d8728a02f0bf2ddc695a8953177f244ab241d7aff6f824564b31c20977d5795d/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/a727b19370308b533c7ae483f112d2b9df700462b2acf5a0af49da74e0b3078b/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-79:{mountpoint:/var/lib/containers/storage/overlay/eca893865c752e71a5cd9e612c4c1af924a227e0af390dccef8a5213a886cc94/merged major:0 minor:79 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/cd5121621b8f78e3ee35aa517116c29f64de62a26912bb3a8297fed211773f08/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/a2f07dfdc3f41c561eb1be5d4808f41a03369ad323422cca9764130a31226e39/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/5221f59f29bc2c4167d422dd76e8f12c9231b9a1506eb328eed57c36d0a7eeea/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/046da112b2f900ea0bb8d1d3966dfd9d295fd7fc770d7108535f0fccb62f3528/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/9c354b236fd232b7add8adef8539080c7500cc9fbb952a94efd66d8ca5f71712/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.486243    4942 manager.go:217] Machine: {Timestamp:2026-11-02 11:56:13.485795359 +0000 UTC m=+0.057145107 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099806 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27576d08b86e6a3c9188c18fdd4451 SystemUUID:ec27576d-08b8-6e6a-3c91-88c18fdd4451 BootID:8f3faf22-f25f-47cb-a978-245137e74cd0 Filesystems:[{Device:overlay_0-151 DeviceMajor:0 DeviceMinor:151 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/691a1fbe-dd29-4579-8e72-8c36c2b1eda2/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:206 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2f3ecd16e39a81d34aa0291317a80fa6fdeeff614be213fcf4c761a24ee92b4e/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e81ecb7e-0219-4976-b881-8c87ab61db50/volumes/kubernetes.io~projected/kube-api-access-7g8ls DeviceMajor:0 DeviceMinor:64 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-138 DeviceMajor:0 DeviceMinor:138 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8d4366f1cd075adb9f1bdcdbdcff5bd0278d670a602795381004153b9933001b/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/0a093470-b06b-49c5-9220-2db08ecfde86/volumes/kubernetes.io~projected/kube-api-access-lbqbh DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2759fb5a-17ad-4c29-b9c8-175769f53069/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-342 DeviceMajor:0 DeviceMinor:342 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e81ecb7e-0219-4976-b881-8c87ab61db50/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7ad8b843416b85c01aba0c78e0dc85ec40f2b20bc8bbc422cd2e3f5f4f0551e9/userdata/shm DeviceMajor:0 DeviceMinor:77 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/81050700-fe2d-45db-b66e-c83400cb2d35/volumes/kubernetes.io~projected/kube-api-access-xgndt DeviceMajor:0 DeviceMinor:221 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a129b2ee37d0619b409638f2e04aad33408dd2dd73f19fec71d9c40bd039ad35/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3794f898cb5401b88d8003f04d64bf4d76af411e8ed96d8cb79881a6bf27ce4f/userdata/shm DeviceMajor:0 DeviceMinor:340 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4/volumes/kubernetes.io~projected/kube-api-access-tbnkn DeviceMajor:0 DeviceMinor:65 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/37670025ba6d0060a48cbaf6065cef83c2fbd01e7a266b2180386d9fa1b75731/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-322 DeviceMajor:0 DeviceMinor:322 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e9dc2cae-7e99-4282-a02d-51fb0f12b43b/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:229 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bbfab1b5-14db-477a-8f06-7cf2c1d058a2/volumes/kubernetes.io~projected/kube-api-access-d6jj5 DeviceMajor:0 DeviceMinor:214 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/fc644b68-76cd-4326-8a15-2f8e3dd393d0/volumes/kubernetes.io~projected/kube-api-access-tvpj5 DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4b5df228477cc031b603768652830fa1d1af537c320e2788ab2e44c2d0663118/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-79 DeviceMajor:0 DeviceMinor:79 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e81ecb7e-0219-4976-b881-8c87ab61db50/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/5e041a46-74d9-4fc1-82e7-6c83d2b3c229/volumes/kubernetes.io~projected/kube-api-access-r554w DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5ca3e4d6-0eac-4309-b049-e5f597728119/volumes/kubernetes.io~projected/kube-api-access-7fndj DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cc29f105-14d5-4004-afe0-1d0c8c17a5fb/volumes/kubernetes.io~projected/kube-api-access-nwnz6 DeviceMajor:0 DeviceMinor:219 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6238f603-d9da-4f8a-98a8-31e69a11b625/volumes/kubernetes.io~projected/kube-api-access-4lh5c DeviceMajor:0 DeviceMinor:339 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/691a1fbe-dd29-4579-8e72-8c36c2b1eda2/volumes/kubernetes.io~projected/kube-api-access-vdzs7 DeviceMajor:0 DeviceMinor:220 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9176bf1e-471e-43a2-af6b-650c5a494efa/volumes/kubernetes.io~projected/kube-api-access-b54rp DeviceMajor:0 DeviceMinor:225 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cb734498-5d2a-40c1-9d1f-443e3e8c8f87/volumes/kubernetes.io~projected/kube-api-access-v7hn7 DeviceMajor:0 DeviceMinor:287 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d33d00be8285b37a83e9f995f598d1de1e53c29edb532922ea2642f91786578b/userdata/shm DeviceMajor:0 DeviceMinor:290 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e6f5f644265046944ccd3504362891bab45948af07c5ceb4eb229f945beb2a60/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a2ccfcfa40dba291334b84b4bc17c12052b8b1d5979211e1e6778c097be27411/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/60dc0a20-c792-4343-aec8-7ff8f86955e9/volumes/kubernetes.io~projected/kube-api-access-tsnt9 DeviceMajor:0 DeviceMinor:63 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1626b253692ce32927cd1c75ee25cbd3815299edb3bb10ccc24efaf9075aa336/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/157531b7c6576657c1682cfa3945f3764b167240c374c1029cdb9badcf0b03e7/userdata/shm DeviceMajor:0 DeviceMinor:211 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/13d85b4b-e889-48b8-97af-f23632533bea/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:210 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/15291745-62ec-45f2-85f9-373504d79de2/volumes/kubernetes.io~projected/kube-api-access-xmwmm DeviceMajor:0 DeviceMinor:218 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e9dc2cae-7e99-4282-a02d-51fb0f12b43b/volumes/kubernetes.io~projected/kube-api-access-vqzcg DeviceMajor:0 DeviceMinor:228 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-288 DeviceMajor:0 DeviceMinor:288 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/528c27028e7efb73e18821bef8122692542cf1cade92a3a969ea56447b01f550/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/98301ff8-eb60-43d4-857f-da7d8da3858b/volumes/kubernetes.io~projected/kube-api-access-2xjgq DeviceMajor:0 DeviceMinor:213 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a05c4de6-e34a-416d-9ed9-35fa6a614dcc/volumes/kubernetes.io~projected/kube-api-access-7hxmh DeviceMajor:0 DeviceMinor:222 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-246 DeviceMajor:0 DeviceMinor:246 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2759fb5a-17ad-4c29-b9c8-175769f53069/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:215 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6384f50d-3bbe-47ef-8e56-5a72d8113c6f/volumes/kubernetes.io~projected/kube-api-access-8v4hm DeviceMajor:0 DeviceMinor:227 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/15291745-62ec-45f2-85f9-373504d79de2/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/13d85b4b-e889-48b8-97af-f23632533bea/volumes/kubernetes.io~projected/kube-api-access-7j7r5 DeviceMajor:0 DeviceMinor:217 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/53ea0d03-1f13-4ee1-af04-9ddf449d6b33/volumes/kubernetes.io~projected/kube-api-access-6zstf DeviceMajor:0 DeviceMinor:223 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f38c9faa-5fae-45ba-93f6-e252b6f092fd/volumes/kubernetes.io~projected/kube-api-access-g9nlh DeviceMajor:0 DeviceMinor:226 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c9f277d97a60b0ae286a17629a77187bc220838b3174007f5e7935e23b963dbc/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2759fb5a-17ad-4c29-b9c8-175769f53069/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/53ea0d03-1f13-4ee1-af04-9ddf449d6b33/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:209 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ba2958d5085f0550a4106550d02a1517e4f0b925a9e2a5a83b34e0bcf8baecb5/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7717a04a35c612622fe1587f3cf2f5d814bab7717ee54f48a2e4a319a83546d5/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/269be49a-6e81-4730-960e-cb0b4a7b031b/volumes/kubernetes.io~projected/kube-api-access-hfp9f DeviceMajor:0 DeviceMinor:224 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-270 DeviceMajor:0 DeviceMinor:270 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6d8ba8e4-34a6-47eb-a33f-b67c52acd079/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:276 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/252df0cc-a5c0-4dea-aca5-bd6b08262d38/volumes/kubernetes.io~projected/kube-api-access-lj85s DeviceMajor:0 DeviceMinor:334 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9176bf1e-471e-43a2-af6b-650c5a494efa/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:205 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-328 DeviceMajor:0 DeviceMinor:328 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6b350fc9-6bb7-43c4-9b89-c5157f42ca26/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e9dc2cae-7e99-4282-a02d-51fb0f12b43b/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-261 DeviceMajor:0 DeviceMinor:261 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-161 DeviceMajor:0 DeviceMinor:161 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2dc02c96-a4d9-46ab-8b52-5382cf43b92c/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/68728c63-ace5-4888-a19b-2782fd2ba673/volumes/kubernetes.io~projected/kube-api-access-hb29w DeviceMajor:0 DeviceMinor:62 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2fad6c8bb3e025df2fcd1778e6bc1e7ad6d715f97a65b8e02e6ca4c254806413/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/300e91d7e2bd6fd9552f384c619772601ce569e29f65a2c676b716953c72a443/userdata/shm DeviceMajor:0 DeviceMinor:295 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9ce07e1cd2e465b2d044ecc7731bc1dba56885c01c9a992463bd713d620cb571/userdata/shm DeviceMajor:0 DeviceMinor:85 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6384f50d-3bbe-47ef-8e56-5a72d8113c6f/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dbd03d2736aff17abe3e92400f910da37ac2f3947dcca36fba592dff611e7de3/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ec3be01ca9e36f5d00ffc5c704ed09be987f8ca357bc7a3e448d221dba99b078/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/4d239eb0-492f-4335-857e-4edc754823e4/volumes/kubernetes.io~projected/kube-api-access-gk6fk DeviceMajor:0 DeviceMinor:61 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/15291745-62ec-45f2-85f9-373504d79de2/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:208 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2dc02c96-a4d9-46ab-8b52-5382cf43b92c/volumes/kubernetes.io~projected/kube-api-access-lbnkr DeviceMajor:0 DeviceMinor:60 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-149 DeviceMajor:0 DeviceMinor:149 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4710cea14069f1ce66928eff09f9ff1fa2ab600549eeb4737a1075940c93d2f0/userdata/shm DeviceMajor:0 DeviceMinor:248 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-164 DeviceMajor:0 DeviceMinor:164 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-274 DeviceMajor:0 DeviceMinor:274 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-281 DeviceMajor:0 DeviceMinor:281 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/2dc02c96-a4d9-46ab-8b52-5382cf43b92c/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e9dc2cae-7e99-4282-a02d-51fb0f12b43b/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:207 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fa8182ba90f40af301e6472b699b54e312b9b5db522fd5e2b3bf7db970775544/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2759fb5a-17ad-4c29-b9c8-175769f53069/volumes/kubernetes.io~projected/kube-api-access-5mg2c DeviceMajor:0 DeviceMinor:216 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:157531b7c657665 MacAddress:d2:12:0c:0a:1a:76 Speed:10000 Mtu:8901} {Name:2f3ecd16e39a81d MacAddress:b2:92:4b:80:65:e1 Speed:10000 Mtu:8901} {Name:2fad6c8bb3e025d MacAddress:5a:85:bd:7b:81:d0 Speed:10000 Mtu:8901} {Name:300e91d7e2bd6fd MacAddress:ae:f9:76:1a:4e:f8 Speed:10000 Mtu:8901} {Name:3794f898cb5401b MacAddress:62:bb:11:b3:76:b0 Speed:10000 Mtu:8901} {Name:4710cea14069f1c MacAddress:72:d9:9e:bb:02:25 Speed:10000 Mtu:8901} {Name:7717a04a35c6126 MacAddress:2a:aa:8c:23:8c:55 Speed:10000 Mtu:8901} {Name:8d4366f1cd075ad MacAddress:42:41:27:33:56:0d Speed:10000 Mtu:8901} {Name:a129b2ee37d0619 MacAddress:02:2a:d9:ce:38:a1 Speed:10000 Mtu:8901} {Name:a2ccfcfa40dba29 MacAddress:66:18:ea:8b:fe:23 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:01:4b:30:61:b7 Speed:0 Mtu:9001} {Name:br-int MacAddress:5e:1c:97:d5:50:2d Speed:0 Mtu:8901} {Name:d33d00be8285b37 MacAddress:be:55:26:33:22:24 Speed:10000 Mtu:8901} {Name:e6f5f6442650469 MacAddress:7e:db:3b:aa:cc:3e Speed:10000 Mtu:8901} {Name:ec3be01ca9e36f5 MacAddress:ae:6f:13:22:55:7a Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:01:4b:30:61:b7 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:da:8b:1b:19:f8:3f Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:2a:08:f7:0a:c2:7e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.486585    4942 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.486626    4942 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.487065    4942 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.487099    4942 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-24.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.487353    4942 topology_manager.go:143] "Creating topology manager with none policy"
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.487366    4942 container_manager_linux.go:311] "Creating device plugin manager"
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.487380    4942 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.487403    4942 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.487429    4942 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.487684    4942 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.487847    4942 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.487890    4942 kubelet.go:493] "Attempting to sync node with API server"
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.487909    4942 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.487924    4942 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.487937    4942 kubelet.go:405] "Adding apiserver pod source"
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.487957    4942 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.489125    4942 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.489142    4942 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 11:56:13.491889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.489681    4942 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 02 11:56:13.493800 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.493695    4942 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 02 11:56:13.493800 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.493727    4942 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 02 11:56:13.494182 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.494168    4942 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 02 11:56:13.494394 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.494379    4942 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 02 11:56:13.494442 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.494412    4942 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 02 11:56:13.494523 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.494514    4942 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 02 11:56:13.494555 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.494528    4942 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 02 11:56:13.494555 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.494535    4942 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 02 11:56:13.494555 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.494541    4942 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 02 11:56:13.494555 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.494547    4942 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 02 11:56:13.494555 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.494552    4942 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 02 11:56:13.494688 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.494558    4942 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 02 11:56:13.494688 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.494563    4942 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 02 11:56:13.494688 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.494571    4942 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 02 11:56:13.494688 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.494577    4942 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 02 11:56:13.494688 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.494586    4942 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 02 11:56:13.494688 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.494595    4942 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 02 11:56:13.494688 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.494620    4942 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 02 11:56:13.494688 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.494626    4942 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 02 11:56:13.497915 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.497904    4942 server.go:1263] "Started kubelet"
Nov 02 11:56:13.498034 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.498005    4942 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 02 11:56:13.498097 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.498003    4942 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 02 11:56:13.498097 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.498074    4942 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 02 11:56:13.499172 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.498411    4942 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 02 11:56:13.498540 ip-10-0-135-24 systemd[1]: Started Kubernetes Kubelet.
Nov 02 11:56:13.503138 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.502053    4942 server.go:317] "Adding debug handlers to kubelet server"
Nov 02 11:56:13.506753 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.504425    4942 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 02 11:56:13.506753 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.504436    4942 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 02 11:56:13.506753 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.504508    4942 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-01 11:50:33 +0000 UTC" deadline="2028-07-26 21:09:40.941195318 +0000 UTC"
Nov 02 11:56:13.506753 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.504547    4942 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15177h13m27.436650879s"
Nov 02 11:56:13.506753 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.504576    4942 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 02 11:56:13.506753 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.504595    4942 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 02 11:56:13.506753 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:13.504612    4942 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-24.us-west-1.compute.internal\" not found"
Nov 02 11:56:13.506753 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.504969    4942 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 02 11:56:13.507116 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.506831    4942 factory.go:55] Registering systemd factory
Nov 02 11:56:13.507116 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.506846    4942 factory.go:223] Registration of the systemd container factory successfully
Nov 02 11:56:13.507334 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.507314    4942 factory.go:153] Registering CRI-O factory
Nov 02 11:56:13.507334 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.507335    4942 factory.go:223] Registration of the crio container factory successfully
Nov 02 11:56:13.507473 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.507396    4942 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 02 11:56:13.507473 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.507419    4942 factory.go:103] Registering Raw factory
Nov 02 11:56:13.507473 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.507434    4942 manager.go:1199] Started watching for new ooms in manager
Nov 02 11:56:13.507848 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.507827    4942 manager.go:319] Starting recovery of all containers
Nov 02 11:56:13.507993 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.507943    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cb734498-5d2a-40c1-9d1f-443e3e8c8f87" volumeName="kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7" seLinuxMountContext=""
Nov 02 11:56:13.507993 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.507986    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d239eb0-492f-4335-857e-4edc754823e4" volumeName="kubernetes.io/empty-dir/4d239eb0-492f-4335-857e-4edc754823e4-tmp-dir" seLinuxMountContext=""
Nov 02 11:56:13.508120 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508015    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53ea0d03-1f13-4ee1-af04-9ddf449d6b33" volumeName="kubernetes.io/configmap/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-trusted-ca" seLinuxMountContext=""
Nov 02 11:56:13.508120 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508031    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="691a1fbe-dd29-4579-8e72-8c36c2b1eda2" volumeName="kubernetes.io/configmap/691a1fbe-dd29-4579-8e72-8c36c2b1eda2-config" seLinuxMountContext=""
Nov 02 11:56:13.508120 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508046    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d8ba8e4-34a6-47eb-a33f-b67c52acd079" volumeName="kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret" seLinuxMountContext=""
Nov 02 11:56:13.508120 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508056    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9dc2cae-7e99-4282-a02d-51fb0f12b43b" volumeName="kubernetes.io/empty-dir/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-ca-trust-extracted" seLinuxMountContext=""
Nov 02 11:56:13.508120 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508065    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2759fb5a-17ad-4c29-b9c8-175769f53069" volumeName="kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-certificates" seLinuxMountContext=""
Nov 02 11:56:13.508120 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508077    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9dc2cae-7e99-4282-a02d-51fb0f12b43b" volumeName="kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-kube-api-access-vqzcg" seLinuxMountContext=""
Nov 02 11:56:13.508120 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508087    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc644b68-76cd-4326-8a15-2f8e3dd393d0" volumeName="kubernetes.io/projected/fc644b68-76cd-4326-8a15-2f8e3dd393d0-kube-api-access-tvpj5" seLinuxMountContext=""
Nov 02 11:56:13.508120 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508096    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5e041a46-74d9-4fc1-82e7-6c83d2b3c229" volumeName="kubernetes.io/projected/5e041a46-74d9-4fc1-82e7-6c83d2b3c229-kube-api-access-r554w" seLinuxMountContext=""
Nov 02 11:56:13.508120 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508105    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6b350fc9-6bb7-43c4-9b89-c5157f42ca26" volumeName="kubernetes.io/secret/6b350fc9-6bb7-43c4-9b89-c5157f42ca26-agent-certs" seLinuxMountContext=""
Nov 02 11:56:13.508120 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508120    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ee9fb0f-d08d-4e09-87bf-df21a5309ca4" volumeName="kubernetes.io/projected/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-kube-api-access-tbnkn" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508131    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c0e3ecc0-e47b-4612-87aa-53ce8434f5a8" volumeName="kubernetes.io/configmap/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-nginx-conf" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508144    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e81ecb7e-0219-4976-b881-8c87ab61db50" volumeName="kubernetes.io/configmap/e81ecb7e-0219-4976-b881-8c87ab61db50-ovnkube-config" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508156    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9dc2cae-7e99-4282-a02d-51fb0f12b43b" volumeName="kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-bound-sa-token" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508165    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13d85b4b-e889-48b8-97af-f23632533bea" volumeName="kubernetes.io/secret/13d85b4b-e889-48b8-97af-f23632533bea-cert" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508174    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2759fb5a-17ad-4c29-b9c8-175769f53069" volumeName="kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-installation-pull-secrets" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508187    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbfab1b5-14db-477a-8f06-7cf2c1d058a2" volumeName="kubernetes.io/empty-dir/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-tmp-dir" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508203    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13d85b4b-e889-48b8-97af-f23632533bea" volumeName="kubernetes.io/projected/13d85b4b-e889-48b8-97af-f23632533bea-kube-api-access-7j7r5" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508213    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="15291745-62ec-45f2-85f9-373504d79de2" volumeName="kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-stats-auth" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508222    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9dc2cae-7e99-4282-a02d-51fb0f12b43b" volumeName="kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-installation-pull-secrets" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508232    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f38c9faa-5fae-45ba-93f6-e252b6f092fd" volumeName="kubernetes.io/configmap/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-config" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508241    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5ca3e4d6-0eac-4309-b049-e5f597728119" volumeName="kubernetes.io/projected/5ca3e4d6-0eac-4309-b049-e5f597728119-kube-api-access-7fndj" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508252    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68728c63-ace5-4888-a19b-2782fd2ba673" volumeName="kubernetes.io/projected/68728c63-ace5-4888-a19b-2782fd2ba673-kube-api-access-hb29w" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508261    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="81050700-fe2d-45db-b66e-c83400cb2d35" volumeName="kubernetes.io/configmap/81050700-fe2d-45db-b66e-c83400cb2d35-telemetry-config" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508278    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9176bf1e-471e-43a2-af6b-650c5a494efa" volumeName="kubernetes.io/empty-dir/9176bf1e-471e-43a2-af6b-650c5a494efa-snapshots" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508339    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a05c4de6-e34a-416d-9ed9-35fa6a614dcc" volumeName="kubernetes.io/projected/a05c4de6-e34a-416d-9ed9-35fa6a614dcc-kube-api-access-7hxmh" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508352    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="15291745-62ec-45f2-85f9-373504d79de2" volumeName="kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-default-certificate" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508381    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2759fb5a-17ad-4c29-b9c8-175769f53069" volumeName="kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-kube-api-access-5mg2c" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508400    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6384f50d-3bbe-47ef-8e56-5a72d8113c6f" volumeName="kubernetes.io/configmap/6384f50d-3bbe-47ef-8e56-5a72d8113c6f-config" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508412    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9176bf1e-471e-43a2-af6b-650c5a494efa" volumeName="kubernetes.io/secret/9176bf1e-471e-43a2-af6b-650c5a494efa-serving-cert" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508424    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbfab1b5-14db-477a-8f06-7cf2c1d058a2" volumeName="kubernetes.io/configmap/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-config-volume" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508434    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="252df0cc-a5c0-4dea-aca5-bd6b08262d38" volumeName="kubernetes.io/projected/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-api-access-lj85s" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508443    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="269be49a-6e81-4730-960e-cb0b4a7b031b" volumeName="kubernetes.io/projected/269be49a-6e81-4730-960e-cb0b4a7b031b-kube-api-access-hfp9f" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508457    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2759fb5a-17ad-4c29-b9c8-175769f53069" volumeName="kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-bound-sa-token" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508476    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2dc02c96-a4d9-46ab-8b52-5382cf43b92c" volumeName="kubernetes.io/projected/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-kube-api-access-lbnkr" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508486    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68728c63-ace5-4888-a19b-2782fd2ba673" volumeName="kubernetes.io/configmap/68728c63-ace5-4888-a19b-2782fd2ba673-cni-binary-copy" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508496    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="691a1fbe-dd29-4579-8e72-8c36c2b1eda2" volumeName="kubernetes.io/projected/691a1fbe-dd29-4579-8e72-8c36c2b1eda2-kube-api-access-vdzs7" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508505    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9176bf1e-471e-43a2-af6b-650c5a494efa" volumeName="kubernetes.io/empty-dir/9176bf1e-471e-43a2-af6b-650c5a494efa-tmp" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508515    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9dc2cae-7e99-4282-a02d-51fb0f12b43b" volumeName="kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-trusted-ca" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508523    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a093470-b06b-49c5-9220-2db08ecfde86" volumeName="kubernetes.io/configmap/0a093470-b06b-49c5-9220-2db08ecfde86-cni-binary-copy" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508534    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2759fb5a-17ad-4c29-b9c8-175769f53069" volumeName="kubernetes.io/empty-dir/2759fb5a-17ad-4c29-b9c8-175769f53069-ca-trust-extracted" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508548    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d239eb0-492f-4335-857e-4edc754823e4" volumeName="kubernetes.io/projected/4d239eb0-492f-4335-857e-4edc754823e4-kube-api-access-gk6fk" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508563    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6384f50d-3bbe-47ef-8e56-5a72d8113c6f" volumeName="kubernetes.io/projected/6384f50d-3bbe-47ef-8e56-5a72d8113c6f-kube-api-access-8v4hm" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508572    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="81050700-fe2d-45db-b66e-c83400cb2d35" volumeName="kubernetes.io/projected/81050700-fe2d-45db-b66e-c83400cb2d35-kube-api-access-xgndt" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508582    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9176bf1e-471e-43a2-af6b-650c5a494efa" volumeName="kubernetes.io/configmap/9176bf1e-471e-43a2-af6b-650c5a494efa-trusted-ca-bundle" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508593    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a093470-b06b-49c5-9220-2db08ecfde86" volumeName="kubernetes.io/configmap/0a093470-b06b-49c5-9220-2db08ecfde86-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508602    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="15291745-62ec-45f2-85f9-373504d79de2" volumeName="kubernetes.io/projected/15291745-62ec-45f2-85f9-373504d79de2-kube-api-access-xmwmm" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508623    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2dc02c96-a4d9-46ab-8b52-5382cf43b92c" volumeName="kubernetes.io/empty-dir/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-tmp" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508639    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68728c63-ace5-4888-a19b-2782fd2ba673" volumeName="kubernetes.io/configmap/68728c63-ace5-4888-a19b-2782fd2ba673-multus-daemon-config" seLinuxMountContext=""
Nov 02 11:56:13.508622 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508648    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6b350fc9-6bb7-43c4-9b89-c5157f42ca26" volumeName="kubernetes.io/configmap/6b350fc9-6bb7-43c4-9b89-c5157f42ca26-konnectivity-ca" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508658    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbfab1b5-14db-477a-8f06-7cf2c1d058a2" volumeName="kubernetes.io/projected/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-kube-api-access-d6jj5" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508665    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc29f105-14d5-4004-afe0-1d0c8c17a5fb" volumeName="kubernetes.io/projected/cc29f105-14d5-4004-afe0-1d0c8c17a5fb-kube-api-access-nwnz6" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508673    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e81ecb7e-0219-4976-b881-8c87ab61db50" volumeName="kubernetes.io/configmap/e81ecb7e-0219-4976-b881-8c87ab61db50-env-overrides" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508681    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2759fb5a-17ad-4c29-b9c8-175769f53069" volumeName="kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-trusted-ca" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508689    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2dc02c96-a4d9-46ab-8b52-5382cf43b92c" volumeName="kubernetes.io/empty-dir/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-tuned" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508707    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9176bf1e-471e-43a2-af6b-650c5a494efa" volumeName="kubernetes.io/configmap/9176bf1e-471e-43a2-af6b-650c5a494efa-service-ca-bundle" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508727    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e81ecb7e-0219-4976-b881-8c87ab61db50" volumeName="kubernetes.io/configmap/e81ecb7e-0219-4976-b881-8c87ab61db50-ovnkube-script-lib" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508734    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e81ecb7e-0219-4976-b881-8c87ab61db50" volumeName="kubernetes.io/projected/e81ecb7e-0219-4976-b881-8c87ab61db50-kube-api-access-7g8ls" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508742    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9dc2cae-7e99-4282-a02d-51fb0f12b43b" volumeName="kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-certificates" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508749    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9dc2cae-7e99-4282-a02d-51fb0f12b43b" volumeName="kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-image-registry-private-configuration" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508757    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc644b68-76cd-4326-8a15-2f8e3dd393d0" volumeName="kubernetes.io/configmap/fc644b68-76cd-4326-8a15-2f8e3dd393d0-serviceca" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508763    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="252df0cc-a5c0-4dea-aca5-bd6b08262d38" volumeName="kubernetes.io/empty-dir/252df0cc-a5c0-4dea-aca5-bd6b08262d38-data-volume" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508771    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53ea0d03-1f13-4ee1-af04-9ddf449d6b33" volumeName="kubernetes.io/configmap/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-config" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508789    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53ea0d03-1f13-4ee1-af04-9ddf449d6b33" volumeName="kubernetes.io/projected/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-kube-api-access-6zstf" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508801    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6238f603-d9da-4f8a-98a8-31e69a11b625" volumeName="kubernetes.io/projected/6238f603-d9da-4f8a-98a8-31e69a11b625-kube-api-access-4lh5c" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508808    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9176bf1e-471e-43a2-af6b-650c5a494efa" volumeName="kubernetes.io/projected/9176bf1e-471e-43a2-af6b-650c5a494efa-kube-api-access-b54rp" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508827    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="98301ff8-eb60-43d4-857f-da7d8da3858b" volumeName="kubernetes.io/projected/98301ff8-eb60-43d4-857f-da7d8da3858b-kube-api-access-2xjgq" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508834    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e81ecb7e-0219-4976-b881-8c87ab61db50" volumeName="kubernetes.io/secret/e81ecb7e-0219-4976-b881-8c87ab61db50-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508841    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2759fb5a-17ad-4c29-b9c8-175769f53069" volumeName="kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-image-registry-private-configuration" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508851    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="53ea0d03-1f13-4ee1-af04-9ddf449d6b33" volumeName="kubernetes.io/secret/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-serving-cert" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508875    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6384f50d-3bbe-47ef-8e56-5a72d8113c6f" volumeName="kubernetes.io/secret/6384f50d-3bbe-47ef-8e56-5a72d8113c6f-serving-cert" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508884    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a093470-b06b-49c5-9220-2db08ecfde86" volumeName="kubernetes.io/configmap/0a093470-b06b-49c5-9220-2db08ecfde86-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508893    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a093470-b06b-49c5-9220-2db08ecfde86" volumeName="kubernetes.io/projected/0a093470-b06b-49c5-9220-2db08ecfde86-kube-api-access-lbqbh" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508900    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5ca3e4d6-0eac-4309-b049-e5f597728119" volumeName="kubernetes.io/configmap/5ca3e4d6-0eac-4309-b049-e5f597728119-iptables-alerter-script" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508931    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f38c9faa-5fae-45ba-93f6-e252b6f092fd" volumeName="kubernetes.io/projected/f38c9faa-5fae-45ba-93f6-e252b6f092fd-kube-api-access-g9nlh" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508941    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="60dc0a20-c792-4343-aec8-7ff8f86955e9" volumeName="kubernetes.io/projected/60dc0a20-c792-4343-aec8-7ff8f86955e9-kube-api-access-tsnt9" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508954    4942 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="691a1fbe-dd29-4579-8e72-8c36c2b1eda2" volumeName="kubernetes.io/secret/691a1fbe-dd29-4579-8e72-8c36c2b1eda2-serving-cert" seLinuxMountContext=""
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508962    4942 reconstruct.go:97] "Volume reconstruction finished"
Nov 02 11:56:13.510432 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.508967    4942 reconciler.go:29] "Reconciler: start to sync state"
Nov 02 11:56:13.531560 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.531532    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/1.log"
Nov 02 11:56:13.532917 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.532897    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/1.log"
Nov 02 11:56:13.533632 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.533601    4942 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 02 11:56:13.535032 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.535004    4942 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 02 11:56:13.535032 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.535036    4942 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 02 11:56:13.535184 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.535064    4942 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 02 11:56:13.535184 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:13.535117    4942 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 02 11:56:13.571680 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.571643    4942 generic.go:356] "Generic (PLEG): container finished" podID="b0871b91e1bc485b0c3cd650ef3a7622" containerID="5bc8b6704f082f422ad11d5fd02c5c8fabe378e5f98f14db16070b7edad24e65" exitCode=0
Nov 02 11:56:13.577724 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.577704    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/1.log"
Nov 02 11:56:13.577839 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.577729    4942 generic.go:356] "Generic (PLEG): container finished" podID="53ea0d03-1f13-4ee1-af04-9ddf449d6b33" containerID="c47445652e8495da228c46a9bb8c8775699f2a05b984ce730a61e30adc8d8cfd" exitCode=255
Nov 02 11:56:13.582712 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.582689    4942 generic.go:356] "Generic (PLEG): container finished" podID="0a093470-b06b-49c5-9220-2db08ecfde86" containerID="48e67b64d1ff53132026a68e8a6ecd5539096a8660ed95b7014b827bed3bee71" exitCode=0
Nov 02 11:56:13.582712 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.582712    4942 generic.go:356] "Generic (PLEG): container finished" podID="0a093470-b06b-49c5-9220-2db08ecfde86" containerID="2fdcbee48ff8fe9f4325e088e7e69be0830fd72dd191e52e20066f6170e6972b" exitCode=0
Nov 02 11:56:13.582871 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.582720    4942 generic.go:356] "Generic (PLEG): container finished" podID="0a093470-b06b-49c5-9220-2db08ecfde86" containerID="ca4ccc893748450eb27ab028d9569c57736e88dd48fcc5160aa4cde1cdeb8c7a" exitCode=0
Nov 02 11:56:13.582871 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.582728    4942 generic.go:356] "Generic (PLEG): container finished" podID="0a093470-b06b-49c5-9220-2db08ecfde86" containerID="ee79522cedf3c4a11e0af63eeb4e9e27033228d8bb535ad52b2845655a286a37" exitCode=0
Nov 02 11:56:13.582871 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.582733    4942 generic.go:356] "Generic (PLEG): container finished" podID="0a093470-b06b-49c5-9220-2db08ecfde86" containerID="f8cbdb591d09fc4d9ebe18dea02164a7295dafb868bf816c5db4a9f7c9c9e36e" exitCode=0
Nov 02 11:56:13.582871 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.582739    4942 generic.go:356] "Generic (PLEG): container finished" podID="0a093470-b06b-49c5-9220-2db08ecfde86" containerID="ec58792d10a1da9b9965785d378713b4f657c9fc0d23da7a3c07d10634c0e946" exitCode=0
Nov 02 11:56:13.599888 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.599869    4942 manager.go:324] Recovery completed
Nov 02 11:56:13.604715 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:13.604663    4942 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-135-24.us-west-1.compute.internal\" not found"
Nov 02 11:56:13.616769 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.616753    4942 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 11:56:13.617986 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.617969    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 11:56:13.618067 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.618000    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 11:56:13.618067 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.618013    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 11:56:13.619905 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.619889    4942 cpu_manager.go:226] "Starting" policy="none"
Nov 02 11:56:13.619905 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.619902    4942 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 02 11:56:13.620011 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.619919    4942 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 02 11:56:13.620011 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.619890    4942 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 11:56:13.620074 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.620037    4942 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 02 11:56:13.620074 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.620047    4942 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 02 11:56:13.620074 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.620063    4942 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 02 11:56:13.620074 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.620069    4942 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 02 11:56:13.620074 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.620074    4942 policy_none.go:50] "Start"
Nov 02 11:56:13.620229 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.620092    4942 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 02 11:56:13.620229 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.620101    4942 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 02 11:56:13.620917 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.620904    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 11:56:13.620991 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.620933    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 11:56:13.620991 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.620947    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 11:56:13.624367 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.624354    4942 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 02 11:56:13.624415 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.624371    4942 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 02 11:56:13.624415 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.624378    4942 policy_none.go:44] "Start"
Nov 02 11:56:13.630110 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.630086    4942 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 02 11:56:13.632819 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.632805    4942 manager.go:341] "Starting Device Plugin manager"
Nov 02 11:56:13.632904 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:13.632831    4942 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 02 11:56:13.632904 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.632841    4942 server.go:87] "Starting device plugin registration server"
Nov 02 11:56:13.633150 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.633136    4942 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 02 11:56:13.633208 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.633149    4942 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 02 11:56:13.633252 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.633234    4942 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 02 11:56:13.633481 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.633375    4942 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 02 11:56:13.633481 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.633420    4942 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 02 11:56:13.633481 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.633426    4942 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 02 11:56:13.633631 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.633479    4942 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-02T11:56:13.633409829Z","UUID":"eeec06c9-2d53-49f4-9052-000814a76be2","Handler":null,"Name":"","Endpoint":""}
Nov 02 11:56:13.635069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.634953    4942 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 02 11:56:13.635069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.634996    4942 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 02 11:56:13.635069 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:13.635014    4942 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 02 11:56:13.635069 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:13.635049    4942 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-24.us-west-1.compute.internal\" not found"
Nov 02 11:56:13.635266 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.635173    4942 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal"]
Nov 02 11:56:13.635350 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.635336    4942 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 11:56:13.636563 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.636546    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 11:56:13.636658 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.636571    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 11:56:13.636658 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.636587    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 11:56:13.636958 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.636940    4942 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 11:56:13.637097 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.637083    4942 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 11:56:13.638580 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.638032    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 11:56:13.638580 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.638047    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 11:56:13.638580 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.638062    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 11:56:13.638580 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.638072    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 11:56:13.638580 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.638078    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 11:56:13.638580 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.638089    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 11:56:13.638580 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:13.638218    4942 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-24.us-west-1.compute.internal\" not found" node="ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:13.638580 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.638412    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal" event={"ID":"a8a4e5ba356fe85cd903457b215689b7","Type":"ContainerStarted","Data":"1bbc335dc329c0c9c4ca960db29a145dcb80a2d64a01aaa20caedf619ac462b8"}
Nov 02 11:56:13.638580 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.638454    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal" event={"ID":"a8a4e5ba356fe85cd903457b215689b7","Type":"ContainerStarted","Data":"fa8182ba90f40af301e6472b699b54e312b9b5db522fd5e2b3bf7db970775544"}
Nov 02 11:56:13.638580 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.638475    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal" event={"ID":"b0871b91e1bc485b0c3cd650ef3a7622","Type":"ContainerStarted","Data":"26287536ef3a282b232fc7ee5b17600e74f1537d737f2b8e643d3253e65fd1fe"}
Nov 02 11:56:13.638580 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.638484    4942 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 11:56:13.638580 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.638493    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal" event={"ID":"b0871b91e1bc485b0c3cd650ef3a7622","Type":"ContainerDied","Data":"5bc8b6704f082f422ad11d5fd02c5c8fabe378e5f98f14db16070b7edad24e65"}
Nov 02 11:56:13.638580 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.638512    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal" event={"ID":"b0871b91e1bc485b0c3cd650ef3a7622","Type":"ContainerStarted","Data":"37670025ba6d0060a48cbaf6065cef83c2fbd01e7a266b2180386d9fa1b75731"}
Nov 02 11:56:13.639401 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.639376    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 11:56:13.639459 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.639404    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 11:56:13.639459 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.639418    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 11:56:13.639555 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:13.639543    4942 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-24.us-west-1.compute.internal\" not found" node="ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:13.654704 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.654687    4942 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 11:56:13.702085 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.702060    4942 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 11:56:13.704208 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.704193    4942 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 11:56:13.706625 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.706608    4942 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:13.733760 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.733741    4942 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 02 11:56:13.735844 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.735829    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 02 11:56:13.735918 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.735858    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 02 11:56:13.735918 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.735868    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 02 11:56:13.735918 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.735886    4942 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:13.754208 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.754189    4942 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 02 11:56:13.754390 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.754376    4942 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 02 11:56:13.766577 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.766561    4942 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 11:56:13.766646 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:13.766602    4942 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:13.766646 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.766615    4942 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:13.824675 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.824654    4942 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 11:56:13.824736 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:13.824692    4942 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:13.852981 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.852964    4942 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:13.853038 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.853020    4942 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:13.853566 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.853552    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeNotReady"
Nov 02 11:56:13.853628 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.853578    4942 setters.go:546] "Node became not ready" node="ip-10-0-135-24.us-west-1.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-02T11:56:13Z","lastTransitionTime":"2026-11-02T11:56:13Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Nov 02 11:56:13.854654 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.854632    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b0871b91e1bc485b0c3cd650ef3a7622-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal\" (UID: \"b0871b91e1bc485b0c3cd650ef3a7622\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:13.854761 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.854667    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0871b91e1bc485b0c3cd650ef3a7622-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal\" (UID: \"b0871b91e1bc485b0c3cd650ef3a7622\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:13.854761 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.854698    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a8a4e5ba356fe85cd903457b215689b7-config\") pod \"kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal\" (UID: \"a8a4e5ba356fe85cd903457b215689b7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:13.867254 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.867230    4942 nodeinfomanager.go:471] existing CSINode "ip-10-0-135-24.us-west-1.compute.internal" is owned by different node (oldNodeID="137cdbc7-0bff-4746-87d9-dbd07fc601d5", newNodeID=""), cleaning up...
Nov 02 11:56:13.955036 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.955014    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b0871b91e1bc485b0c3cd650ef3a7622-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal\" (UID: \"b0871b91e1bc485b0c3cd650ef3a7622\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:13.955167 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.955051    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0871b91e1bc485b0c3cd650ef3a7622-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal\" (UID: \"b0871b91e1bc485b0c3cd650ef3a7622\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:13.955167 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.955111    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a8a4e5ba356fe85cd903457b215689b7-config\") pod \"kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal\" (UID: \"a8a4e5ba356fe85cd903457b215689b7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:13.955167 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.955124    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b0871b91e1bc485b0c3cd650ef3a7622-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal\" (UID: \"b0871b91e1bc485b0c3cd650ef3a7622\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:13.955167 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.955158    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a8a4e5ba356fe85cd903457b215689b7-config\") pod \"kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal\" (UID: \"a8a4e5ba356fe85cd903457b215689b7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:13.955328 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:13.955191    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b0871b91e1bc485b0c3cd650ef3a7622-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal\" (UID: \"b0871b91e1bc485b0c3cd650ef3a7622\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:14.124331 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.124241    4942 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-135-24.us-west-1.compute.internal" event="NodeReady"
Nov 02 11:56:14.124479 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.124374    4942 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 02 11:56:14.488490 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.488420    4942 apiserver.go:52] "Watching apiserver"
Nov 02 11:56:14.587837 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.587812    4942 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:14.587971 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.587950    4942 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:14.641185 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.641158    4942 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 02 11:56:14.641862 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.641839    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d58f4b759-ts6c2","openshift-ingress/router-default-567988bb57-rzsqv","openshift-multus/multus-additional-cni-plugins-mwslp","openshift-console-operator/console-operator-7cd56b8dbc-2qq64","openshift-multus/multus-fsq7c","openshift-insights/insights-operator-7c4986d449-rhl9j","openshift-network-console/networking-console-plugin-86c478bb88-hflkz","openshift-network-operator/iptables-alerter-dqg86","openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77","openshift-dns/dns-default-cttl7","openshift-network-diagnostics/network-check-target-2vw9j","openshift-image-registry/node-ca-xt7dv","openshift-insights/insights-runtime-extractor-krn29","kube-system/konnectivity-agent-797nn","openshift-cluster-node-tuning-operator/tuned-sv47s","openshift-ingress-canary/ingress-canary-tlvw4","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm","openshift-dns/node-resolver-h24nt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal","openshift-multus/network-metrics-daemon-n4tth","kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9pfhc","openshift-ovn-kubernetes/ovnkube-node-7686t","kube-system/kas-connection-checker-685895c7c9-lskz4","kube-system/kas-connection-checker-685895c7c9-sgrg9","openshift-network-diagnostics/network-check-source-6f78d59f99-g7gnd","openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4","openshift-kube-storage-version-migrator/migrator-66df5d496b-gdvmz","kube-system/kas-connection-checker-685895c7c9-7tt7c","kube-system/global-pull-secret-syncer-vm62x","openshift-image-registry/image-registry-75766bbfd6-gpgxx","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"]
Nov 02 11:56:14.643057 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.643044    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:14.643355 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.643334    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:14.644373 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.644348    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:14.644816 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.644800    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:14.649367 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.646903    4942 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 11:56:14.649367 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.646954    4942 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:14.649367 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.647435    4942 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 02 11:56:14.649367 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.647491    4942 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-24.us-west-1.compute.internal"
Nov 02 11:56:14.649367 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.647884    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:14.649367 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.648651    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:56:14.654380 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.654361    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:14.655916 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.655895    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:14.656584 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.656567    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:14.695794 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.695774    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 02 11:56:14.695891 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.695876    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 02 11:56:14.696229 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.696210    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 02 11:56:14.696229 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.696216    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 02 11:56:14.696358 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.696242    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 02 11:56:14.696358 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.696234    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 02 11:56:14.696473 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.696458    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 02 11:56:14.696526 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.696479    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 02 11:56:14.696526 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.696508    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 02 11:56:14.696663 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.696652    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 02 11:56:14.696855 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.696841    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 02 11:56:14.697683 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.697663    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.697781 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.697719    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 02 11:56:14.697781 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.697721    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.697781 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.697739    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.697937 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.697894    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 02 11:56:14.697937 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.697930    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.698026 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.698008    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.698395 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.698377    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 02 11:56:14.698547 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.698532    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 02 11:56:14.698663 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.698647    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.698729 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.698649    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 02 11:56:14.698729 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.698709    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.698729 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.698711    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.698871 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.698856    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.698965 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.698950    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.699019 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.698995    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.699019 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.699007    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.699231 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.699205    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.699231 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.699227    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.699405 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.699272    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.699405 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.699275    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 02 11:56:14.699405 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.699388    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 02 11:56:14.699537 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.699461    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.699537 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.699475    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.699537 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.699490    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 02 11:56:14.699537 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.699519    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.699732 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.699559    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.699732 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.699676    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 02 11:56:14.707584 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.707563    4942 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 02 11:56:14.749702 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.749639    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 02 11:56:14.750243 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.750222    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.750386 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.750369    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 02 11:56:14.750500 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.750484    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.750570 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.750515    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-msdx6\""
Nov 02 11:56:14.750656 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.750643    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 02 11:56:14.750902 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.750885    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-bjwgn\""
Nov 02 11:56:14.751345 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.751328    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-lk8z8\""
Nov 02 11:56:14.751468 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.751451    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x7vnq\""
Nov 02 11:56:14.751676 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.751660    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.751739 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.751675    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 02 11:56:14.751783 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.751741    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.751826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.751799    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hx94g\""
Nov 02 11:56:14.751880 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.751862    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 02 11:56:14.752041 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.752023    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.752208 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.752191    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 02 11:56:14.752339 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.752320    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-g4drg\""
Nov 02 11:56:14.752513 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.752495    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 02 11:56:14.752621 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.752605    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.752762 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.752744    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.752826 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.752778    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gpdd6\""
Nov 02 11:56:14.752953 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.752934    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.753126 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753109    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 02 11:56:14.753181 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753164    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 02 11:56:14.753271 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753254    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qcv25\""
Nov 02 11:56:14.753354 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753309    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.753525 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753503    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-hqqr9\""
Nov 02 11:56:14.753581 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753539    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-q864t\""
Nov 02 11:56:14.753581 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753559    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.753663 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753589    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.753663 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753618    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 02 11:56:14.753785 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753669    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.753785 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753716    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.753893 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753808    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 02 11:56:14.753893 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753837    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 02 11:56:14.753893 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753860    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.754050 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753909    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 02 11:56:14.754050 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753944    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 02 11:56:14.754050 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753972    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2rrmx\""
Nov 02 11:56:14.754050 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753975    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gfx2g\""
Nov 02 11:56:14.754050 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.754051    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 02 11:56:14.754287 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.754071    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 02 11:56:14.754287 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.754164    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 02 11:56:14.754287 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.754185    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 02 11:56:14.754442 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.753947    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 02 11:56:14.754500 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.754470    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.754500 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.754494    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.754635 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.754598    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.754635 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.754601    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4pvvq\""
Nov 02 11:56:14.754761 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.754747    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 02 11:56:14.755221 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.755207    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-n2k42\""
Nov 02 11:56:14.755329 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.755292    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 02 11:56:14.755403 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.755344    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 02 11:56:14.755403 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.755394    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-hc67r\""
Nov 02 11:56:14.755515 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.755432    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7p526\""
Nov 02 11:56:14.755515 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.755473    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4wp8v\""
Nov 02 11:56:14.755635 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.755571    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 02 11:56:14.755635 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.755576    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 02 11:56:14.755635 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.755602    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mvmxd\""
Nov 02 11:56:14.755635 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.755603    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 02 11:56:14.755635 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.755605    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 02 11:56:14.760678 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.760663    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a093470-b06b-49c5-9220-2db08ecfde86-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:56:14.760730 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.760687    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/691a1fbe-dd29-4579-8e72-8c36c2b1eda2-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf\" (UID: \"691a1fbe-dd29-4579-8e72-8c36c2b1eda2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"
Nov 02 11:56:14.760730 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.760706    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d239eb0-492f-4335-857e-4edc754823e4-hosts-file\") pod \"node-resolver-h24nt\" (UID: \"4d239eb0-492f-4335-857e-4edc754823e4\") " pod="openshift-dns/node-resolver-h24nt"
Nov 02 11:56:14.760828 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.760725    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-socket-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.760828 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.760752    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-trusted-ca\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:14.760918 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.760863    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:14.760962 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.760927    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-slash\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.760962 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.760951    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-run-openvswitch\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.761077 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761055    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbnkn\" (UniqueName: \"kubernetes.io/projected/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-kube-api-access-tbnkn\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.761077 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761067    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/691a1fbe-dd29-4579-8e72-8c36c2b1eda2-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf\" (UID: \"691a1fbe-dd29-4579-8e72-8c36c2b1eda2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"
Nov 02 11:56:14.761175 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761101    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b54rp\" (UniqueName: \"kubernetes.io/projected/9176bf1e-471e-43a2-af6b-650c5a494efa-kube-api-access-b54rp\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:14.761175 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761134    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a093470-b06b-49c5-9220-2db08ecfde86-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:56:14.761237 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761214    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:14.761326 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761283    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-trusted-ca\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:14.761360 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761322    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lh5c\" (UniqueName: \"kubernetes.io/projected/6238f603-d9da-4f8a-98a8-31e69a11b625-kube-api-access-4lh5c\") pod \"migrator-66df5d496b-gdvmz\" (UID: \"6238f603-d9da-4f8a-98a8-31e69a11b625\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gdvmz"
Nov 02 11:56:14.761494 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761483    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-tuned\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.761529 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761502    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9176bf1e-471e-43a2-af6b-650c5a494efa-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:14.761529 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761519    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9176bf1e-471e-43a2-af6b-650c5a494efa-service-ca-bundle\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:14.761587 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761535    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-certificates\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:14.761587 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761569    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:14.761651 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761596    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-image-registry-private-configuration\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:14.761651 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761613    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-run-netns\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.761651 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761622    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-tuned\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.761784 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761665    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:14.761784 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761683    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9176bf1e-471e-43a2-af6b-650c5a494efa-service-ca-bundle\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:14.761784 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761713    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6jj5\" (UniqueName: \"kubernetes.io/projected/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-kube-api-access-d6jj5\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:14.761784 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761745    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:14.761784 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761777    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e81ecb7e-0219-4976-b881-8c87ab61db50-ovnkube-script-lib\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.762020 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761803    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13d85b4b-e889-48b8-97af-f23632533bea-cert\") pod \"ingress-canary-tlvw4\" (UID: \"13d85b4b-e889-48b8-97af-f23632533bea\") " pod="openshift-ingress-canary/ingress-canary-tlvw4"
Nov 02 11:56:14.762020 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761829    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fndj\" (UniqueName: \"kubernetes.io/projected/5ca3e4d6-0eac-4309-b049-e5f597728119-kube-api-access-7fndj\") pod \"iptables-alerter-dqg86\" (UID: \"5ca3e4d6-0eac-4309-b049-e5f597728119\") " pod="openshift-network-operator/iptables-alerter-dqg86"
Nov 02 11:56:14.762020 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761829    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-image-registry-private-configuration\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:14.762020 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761837    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-certificates\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:14.762020 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761853    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/68728c63-ace5-4888-a19b-2782fd2ba673-multus-daemon-config\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.762020 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761893    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-var-lib-kubelet\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.762020 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761913    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9176bf1e-471e-43a2-af6b-650c5a494efa-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:14.762020 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761938    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-installation-pull-secrets\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:14.762020 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761966    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmwmm\" (UniqueName: \"kubernetes.io/projected/15291745-62ec-45f2-85f9-373504d79de2-kube-api-access-xmwmm\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:14.762020 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.761991    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk6fk\" (UniqueName: \"kubernetes.io/projected/4d239eb0-492f-4335-857e-4edc754823e4-kube-api-access-gk6fk\") pod \"node-resolver-h24nt\" (UID: \"4d239eb0-492f-4335-857e-4edc754823e4\") " pod="openshift-dns/node-resolver-h24nt"
Nov 02 11:56:14.762020 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762023    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j7r5\" (UniqueName: \"kubernetes.io/projected/13d85b4b-e889-48b8-97af-f23632533bea-kube-api-access-7j7r5\") pod \"ingress-canary-tlvw4\" (UID: \"13d85b4b-e889-48b8-97af-f23632533bea\") " pod="openshift-ingress-canary/ingress-canary-tlvw4"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762049    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-var-lib-cni-bin\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762077    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-modprobe-d\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762100    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-systemd\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762126    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-ca-trust-extracted\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762050    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13d85b4b-e889-48b8-97af-f23632533bea-cert\") pod \"ingress-canary-tlvw4\" (UID: \"13d85b4b-e889-48b8-97af-f23632533bea\") " pod="openshift-ingress-canary/ingress-canary-tlvw4"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762154    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zstf\" (UniqueName: \"kubernetes.io/projected/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-kube-api-access-6zstf\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762164    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/68728c63-ace5-4888-a19b-2782fd2ba673-multus-daemon-config\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762205    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-installation-pull-secrets\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762236    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e81ecb7e-0219-4976-b881-8c87ab61db50-ovnkube-script-lib\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762241    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-ca-trust-extracted\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762312    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfp9f\" (UniqueName: \"kubernetes.io/projected/269be49a-6e81-4730-960e-cb0b4a7b031b-kube-api-access-hfp9f\") pod \"kas-connection-checker-685895c7c9-lskz4\" (UID: \"269be49a-6e81-4730-960e-cb0b4a7b031b\") " pod="kube-system/kas-connection-checker-685895c7c9-lskz4"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762336    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762351    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-run-k8s-cni-cncf-io\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762366    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-cni-netd\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762403    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hxmh\" (UniqueName: \"kubernetes.io/projected/a05c4de6-e34a-416d-9ed9-35fa6a614dcc-kube-api-access-7hxmh\") pod \"volume-data-source-validator-58c9f5f46d-9pfhc\" (UID: \"a05c4de6-e34a-416d-9ed9-35fa6a614dcc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9pfhc"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762427    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-tmp\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762477    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-tmp\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762503    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-run-netns\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.762524 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762534    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-node-log\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762552    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-installation-pull-secrets\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762591    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvpj5\" (UniqueName: \"kubernetes.io/projected/fc644b68-76cd-4326-8a15-2f8e3dd393d0-kube-api-access-tvpj5\") pod \"node-ca-xt7dv\" (UID: \"fc644b68-76cd-4326-8a15-2f8e3dd393d0\") " pod="openshift-image-registry/node-ca-xt7dv"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762662    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a093470-b06b-49c5-9220-2db08ecfde86-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762686    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-installation-pull-secrets\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762706    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762735    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-var-lib-cni-multus\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762759    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-etc-kubernetes\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762784    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb29w\" (UniqueName: \"kubernetes.io/projected/68728c63-ace5-4888-a19b-2782fd2ba673-kube-api-access-hb29w\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762819    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a093470-b06b-49c5-9220-2db08ecfde86-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762823    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-sys\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.762917    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-trusted-ca\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763035    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-original-pull-secret\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763045    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-dbus\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763096    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d239eb0-492f-4335-857e-4edc754823e4-tmp-dir\") pod \"node-resolver-h24nt\" (UID: \"4d239eb0-492f-4335-857e-4edc754823e4\") " pod="openshift-dns/node-resolver-h24nt"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763131    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-host\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763184    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-image-registry-private-configuration\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763192    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d239eb0-492f-4335-857e-4edc754823e4-tmp-dir\") pod \"node-resolver-h24nt\" (UID: \"4d239eb0-492f-4335-857e-4edc754823e4\") " pod="openshift-dns/node-resolver-h24nt"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763212    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a093470-b06b-49c5-9220-2db08ecfde86-system-cni-dir\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763241    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9nlh\" (UniqueName: \"kubernetes.io/projected/f38c9faa-5fae-45ba-93f6-e252b6f092fd-kube-api-access-g9nlh\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:14.763262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763251    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-trusted-ca\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763285    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/252df0cc-a5c0-4dea-aca5-bd6b08262d38-data-volume\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763326    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6384f50d-3bbe-47ef-8e56-5a72d8113c6f-config\") pod \"service-ca-operator-544d749c8c-9mfg4\" (UID: \"6384f50d-3bbe-47ef-8e56-5a72d8113c6f\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763352    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-image-registry-private-configuration\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763352    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-bound-sa-token\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763391    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a093470-b06b-49c5-9220-2db08ecfde86-cnibin\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763417    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-trusted-ca\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763438    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/252df0cc-a5c0-4dea-aca5-bd6b08262d38-data-volume\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763440    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-stats-auth\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763480    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mg2c\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-kube-api-access-5mg2c\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763522    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6384f50d-3bbe-47ef-8e56-5a72d8113c6f-config\") pod \"service-ca-operator-544d749c8c-9mfg4\" (UID: \"6384f50d-3bbe-47ef-8e56-5a72d8113c6f\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763555    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-config-volume\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763579    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/68728c63-ace5-4888-a19b-2782fd2ba673-cni-binary-copy\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763624    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-multus-conf-dir\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763640    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-stats-auth\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763649    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-run-systemd\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763683    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwnz6\" (UniqueName: \"kubernetes.io/projected/cc29f105-14d5-4004-afe0-1d0c8c17a5fb-kube-api-access-nwnz6\") pod \"kas-connection-checker-685895c7c9-7tt7c\" (UID: \"cc29f105-14d5-4004-afe0-1d0c8c17a5fb\") " pod="kube-system/kas-connection-checker-685895c7c9-7tt7c"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763707    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9176bf1e-471e-43a2-af6b-650c5a494efa-serving-cert\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763711    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-trusted-ca\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763722    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/691a1fbe-dd29-4579-8e72-8c36c2b1eda2-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf\" (UID: \"691a1fbe-dd29-4579-8e72-8c36c2b1eda2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763768    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763779    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-config-volume\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763793    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763807    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/68728c63-ace5-4888-a19b-2782fd2ba673-cni-binary-copy\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763823    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-kubernetes\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763857    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9176bf1e-471e-43a2-af6b-650c5a494efa-serving-cert\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763903    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-run\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763917    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/691a1fbe-dd29-4579-8e72-8c36c2b1eda2-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf\" (UID: \"691a1fbe-dd29-4579-8e72-8c36c2b1eda2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763939    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-bound-sa-token\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763979    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a093470-b06b-49c5-9220-2db08ecfde86-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.763995    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-nginx-conf\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764042    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-certificates\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:14.764069 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764068    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6b350fc9-6bb7-43c4-9b89-c5157f42ca26-agent-certs\") pod \"konnectivity-agent-797nn\" (UID: \"6b350fc9-6bb7-43c4-9b89-c5157f42ca26\") " pod="kube-system/konnectivity-agent-797nn"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764095    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9176bf1e-471e-43a2-af6b-650c5a494efa-tmp\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764209    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-nginx-conf\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764214    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a093470-b06b-49c5-9220-2db08ecfde86-cni-binary-copy\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764222    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9176bf1e-471e-43a2-af6b-650c5a494efa-tmp\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764244    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-tmp-dir\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764270    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-var-lib-kubelet\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764281    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-certificates\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764286    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-run-ovn\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764330    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e81ecb7e-0219-4976-b881-8c87ab61db50-env-overrides\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764349    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e81ecb7e-0219-4976-b881-8c87ab61db50-ovn-node-metrics-cert\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764364    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764363    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-tmp-dir\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764365    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a093470-b06b-49c5-9220-2db08ecfde86-cni-binary-copy\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764402    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-device-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764405    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6b350fc9-6bb7-43c4-9b89-c5157f42ca26-agent-certs\") pod \"konnectivity-agent-797nn\" (UID: \"6b350fc9-6bb7-43c4-9b89-c5157f42ca26\") " pod="kube-system/konnectivity-agent-797nn"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764483    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e81ecb7e-0219-4976-b881-8c87ab61db50-env-overrides\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764494    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-sys-fs\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764491    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e81ecb7e-0219-4976-b881-8c87ab61db50-ovn-node-metrics-cert\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764518    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsnt9\" (UniqueName: \"kubernetes.io/projected/60dc0a20-c792-4343-aec8-7ff8f86955e9-kube-api-access-tsnt9\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764550    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6384f50d-3bbe-47ef-8e56-5a72d8113c6f-serving-cert\") pod \"service-ca-operator-544d749c8c-9mfg4\" (UID: \"6384f50d-3bbe-47ef-8e56-5a72d8113c6f\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764569    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-multus-cni-dir\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764661    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-cni-bin\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764677    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5ca3e4d6-0eac-4309-b049-e5f597728119-iptables-alerter-script\") pod \"iptables-alerter-dqg86\" (UID: \"5ca3e4d6-0eac-4309-b049-e5f597728119\") " pod="openshift-network-operator/iptables-alerter-dqg86"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764703    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764735    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-system-cni-dir\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764764    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-sysconfig\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764786    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-log-socket\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764806    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-run-ovn-kubernetes\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764824    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6384f50d-3bbe-47ef-8e56-5a72d8113c6f-serving-cert\") pod \"service-ca-operator-544d749c8c-9mfg4\" (UID: \"6384f50d-3bbe-47ef-8e56-5a72d8113c6f\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764832    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fc644b68-76cd-4326-8a15-2f8e3dd393d0-serviceca\") pod \"node-ca-xt7dv\" (UID: \"fc644b68-76cd-4326-8a15-2f8e3dd393d0\") " pod="openshift-image-registry/node-ca-xt7dv"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764896    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdzs7\" (UniqueName: \"kubernetes.io/projected/691a1fbe-dd29-4579-8e72-8c36c2b1eda2-kube-api-access-vdzs7\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf\" (UID: \"691a1fbe-dd29-4579-8e72-8c36c2b1eda2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764915    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5ca3e4d6-0eac-4309-b049-e5f597728119-iptables-alerter-script\") pod \"iptables-alerter-dqg86\" (UID: \"5ca3e4d6-0eac-4309-b049-e5f597728119\") " pod="openshift-network-operator/iptables-alerter-dqg86"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764936    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764967    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fc644b68-76cd-4326-8a15-2f8e3dd393d0-serviceca\") pod \"node-ca-xt7dv\" (UID: \"fc644b68-76cd-4326-8a15-2f8e3dd393d0\") " pod="openshift-image-registry/node-ca-xt7dv"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.764987    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r554w\" (UniqueName: \"kubernetes.io/projected/5e041a46-74d9-4fc1-82e7-6c83d2b3c229-kube-api-access-r554w\") pod \"kas-connection-checker-685895c7c9-sgrg9\" (UID: \"5e041a46-74d9-4fc1-82e7-6c83d2b3c229\") " pod="kube-system/kas-connection-checker-685895c7c9-sgrg9"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765022    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-run-multus-certs\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765039    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7hn7\" (UniqueName: \"kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7\") pod \"network-check-target-2vw9j\" (UID: \"cb734498-5d2a-40c1-9d1f-443e3e8c8f87\") " pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765096    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9176bf1e-471e-43a2-af6b-650c5a494efa-snapshots\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765124    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765175    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v4hm\" (UniqueName: \"kubernetes.io/projected/6384f50d-3bbe-47ef-8e56-5a72d8113c6f-kube-api-access-8v4hm\") pod \"service-ca-operator-544d749c8c-9mfg4\" (UID: \"6384f50d-3bbe-47ef-8e56-5a72d8113c6f\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"
Nov 02 11:56:14.765177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765211    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-cnibin\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765226    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9176bf1e-471e-43a2-af6b-650c5a494efa-snapshots\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765333    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-kubelet\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765369    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e81ecb7e-0219-4976-b881-8c87ab61db50-ovnkube-config\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765427    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqzcg\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-kube-api-access-vqzcg\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765475    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-registration-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765568    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e81ecb7e-0219-4976-b881-8c87ab61db50-ovnkube-config\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765587    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbqbh\" (UniqueName: \"kubernetes.io/projected/0a093470-b06b-49c5-9220-2db08ecfde86-kube-api-access-lbqbh\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765616    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lj85s\" (UniqueName: \"kubernetes.io/projected/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-api-access-lj85s\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765660    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-hostroot\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765709    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbnkr\" (UniqueName: \"kubernetes.io/projected/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-kube-api-access-lbnkr\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765818    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-sysctl-conf\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765850    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-etc-selinux\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765873    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2759fb5a-17ad-4c29-b9c8-175769f53069-ca-trust-extracted\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765936    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2759fb5a-17ad-4c29-b9c8-175769f53069-ca-trust-extracted\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765956    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.765987    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6b350fc9-6bb7-43c4-9b89-c5157f42ca26-konnectivity-ca\") pod \"konnectivity-agent-797nn\" (UID: \"6b350fc9-6bb7-43c4-9b89-c5157f42ca26\") " pod="kube-system/konnectivity-agent-797nn"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766003    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-config\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766096    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgndt\" (UniqueName: \"kubernetes.io/projected/81050700-fe2d-45db-b66e-c83400cb2d35-kube-api-access-xgndt\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766122    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xjgq\" (UniqueName: \"kubernetes.io/projected/98301ff8-eb60-43d4-857f-da7d8da3858b-kube-api-access-2xjgq\") pod \"network-check-source-6f78d59f99-g7gnd\" (UID: \"98301ff8-eb60-43d4-857f-da7d8da3858b\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-g7gnd"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766142    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-etc-openvswitch\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766145    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-config\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766207    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc644b68-76cd-4326-8a15-2f8e3dd393d0-host\") pod \"node-ca-xt7dv\" (UID: \"fc644b68-76cd-4326-8a15-2f8e3dd393d0\") " pod="openshift-image-registry/node-ca-xt7dv"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766226    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6b350fc9-6bb7-43c4-9b89-c5157f42ca26-konnectivity-ca\") pod \"konnectivity-agent-797nn\" (UID: \"6b350fc9-6bb7-43c4-9b89-c5157f42ca26\") " pod="kube-system/konnectivity-agent-797nn"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766243    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-default-certificate\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766281    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-sysctl-d\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766332    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g8ls\" (UniqueName: \"kubernetes.io/projected/e81ecb7e-0219-4976-b881-8c87ab61db50-kube-api-access-7g8ls\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766361    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766377    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-kubelet-config\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766406    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-lib-modules\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766449    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-default-certificate\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766466    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-serving-cert\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766492    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/252df0cc-a5c0-4dea-aca5-bd6b08262d38-crio-socket\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766543    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ca3e4d6-0eac-4309-b049-e5f597728119-host-slash\") pod \"iptables-alerter-dqg86\" (UID: \"5ca3e4d6-0eac-4309-b049-e5f597728119\") " pod="openshift-network-operator/iptables-alerter-dqg86"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766585    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-serving-cert\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766588    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/81050700-fe2d-45db-b66e-c83400cb2d35-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766616    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-multus-socket-dir-parent\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766688    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-systemd-units\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766720    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-var-lib-openvswitch\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766740    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:14.766763 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.766760    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/81050700-fe2d-45db-b66e-c83400cb2d35-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:14.821105 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.821074    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b54rp\" (UniqueName: \"kubernetes.io/projected/9176bf1e-471e-43a2-af6b-650c5a494efa-kube-api-access-b54rp\") pod \"insights-operator-7c4986d449-rhl9j\" (UID: \"9176bf1e-471e-43a2-af6b-650c5a494efa\") " pod="openshift-insights/insights-operator-7c4986d449-rhl9j"
Nov 02 11:56:14.821105 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.821089    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lh5c\" (UniqueName: \"kubernetes.io/projected/6238f603-d9da-4f8a-98a8-31e69a11b625-kube-api-access-4lh5c\") pod \"migrator-66df5d496b-gdvmz\" (UID: \"6238f603-d9da-4f8a-98a8-31e69a11b625\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-gdvmz"
Nov 02 11:56:14.825495 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.825044    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbnkn\" (UniqueName: \"kubernetes.io/projected/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-kube-api-access-tbnkn\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.825495 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.825205    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvpj5\" (UniqueName: \"kubernetes.io/projected/fc644b68-76cd-4326-8a15-2f8e3dd393d0-kube-api-access-tvpj5\") pod \"node-ca-xt7dv\" (UID: \"fc644b68-76cd-4326-8a15-2f8e3dd393d0\") " pod="openshift-image-registry/node-ca-xt7dv"
Nov 02 11:56:14.825495 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.825293    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-bound-sa-token\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:14.825495 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.825330    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsnt9\" (UniqueName: \"kubernetes.io/projected/60dc0a20-c792-4343-aec8-7ff8f86955e9-kube-api-access-tsnt9\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:56:14.825495 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.825387    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwnz6\" (UniqueName: \"kubernetes.io/projected/cc29f105-14d5-4004-afe0-1d0c8c17a5fb-kube-api-access-nwnz6\") pod \"kas-connection-checker-685895c7c9-7tt7c\" (UID: \"cc29f105-14d5-4004-afe0-1d0c8c17a5fb\") " pod="kube-system/kas-connection-checker-685895c7c9-7tt7c"
Nov 02 11:56:14.825495 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.825478    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb29w\" (UniqueName: \"kubernetes.io/projected/68728c63-ace5-4888-a19b-2782fd2ba673-kube-api-access-hb29w\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.825764 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.825616    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9nlh\" (UniqueName: \"kubernetes.io/projected/f38c9faa-5fae-45ba-93f6-e252b6f092fd-kube-api-access-g9nlh\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:14.825815 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.825779    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk6fk\" (UniqueName: \"kubernetes.io/projected/4d239eb0-492f-4335-857e-4edc754823e4-kube-api-access-gk6fk\") pod \"node-resolver-h24nt\" (UID: \"4d239eb0-492f-4335-857e-4edc754823e4\") " pod="openshift-dns/node-resolver-h24nt"
Nov 02 11:56:14.825856 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.825822    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fndj\" (UniqueName: \"kubernetes.io/projected/5ca3e4d6-0eac-4309-b049-e5f597728119-kube-api-access-7fndj\") pod \"iptables-alerter-dqg86\" (UID: \"5ca3e4d6-0eac-4309-b049-e5f597728119\") " pod="openshift-network-operator/iptables-alerter-dqg86"
Nov 02 11:56:14.825984 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.825960    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mg2c\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-kube-api-access-5mg2c\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:14.826072 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.825965    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hxmh\" (UniqueName: \"kubernetes.io/projected/a05c4de6-e34a-416d-9ed9-35fa6a614dcc-kube-api-access-7hxmh\") pod \"volume-data-source-validator-58c9f5f46d-9pfhc\" (UID: \"a05c4de6-e34a-416d-9ed9-35fa6a614dcc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-9pfhc"
Nov 02 11:56:14.826072 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.826055    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j7r5\" (UniqueName: \"kubernetes.io/projected/13d85b4b-e889-48b8-97af-f23632533bea-kube-api-access-7j7r5\") pod \"ingress-canary-tlvw4\" (UID: \"13d85b4b-e889-48b8-97af-f23632533bea\") " pod="openshift-ingress-canary/ingress-canary-tlvw4"
Nov 02 11:56:14.826183 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.826168    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-bound-sa-token\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:14.830865 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.830735    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r554w\" (UniqueName: \"kubernetes.io/projected/5e041a46-74d9-4fc1-82e7-6c83d2b3c229-kube-api-access-r554w\") pod \"kas-connection-checker-685895c7c9-sgrg9\" (UID: \"5e041a46-74d9-4fc1-82e7-6c83d2b3c229\") " pod="kube-system/kas-connection-checker-685895c7c9-sgrg9"
Nov 02 11:56:14.830865 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.830743    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdzs7\" (UniqueName: \"kubernetes.io/projected/691a1fbe-dd29-4579-8e72-8c36c2b1eda2-kube-api-access-vdzs7\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf\" (UID: \"691a1fbe-dd29-4579-8e72-8c36c2b1eda2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf"
Nov 02 11:56:14.830865 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.830806    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfp9f\" (UniqueName: \"kubernetes.io/projected/269be49a-6e81-4730-960e-cb0b4a7b031b-kube-api-access-hfp9f\") pod \"kas-connection-checker-685895c7c9-lskz4\" (UID: \"269be49a-6e81-4730-960e-cb0b4a7b031b\") " pod="kube-system/kas-connection-checker-685895c7c9-lskz4"
Nov 02 11:56:14.831078 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.830861    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6jj5\" (UniqueName: \"kubernetes.io/projected/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-kube-api-access-d6jj5\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:14.831078 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.830911    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7hn7\" (UniqueName: \"kubernetes.io/projected/cb734498-5d2a-40c1-9d1f-443e3e8c8f87-kube-api-access-v7hn7\") pod \"network-check-target-2vw9j\" (UID: \"cb734498-5d2a-40c1-9d1f-443e3e8c8f87\") " pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:56:14.831078 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.831003    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmwmm\" (UniqueName: \"kubernetes.io/projected/15291745-62ec-45f2-85f9-373504d79de2-kube-api-access-xmwmm\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:14.831280 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.831259    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbqbh\" (UniqueName: \"kubernetes.io/projected/0a093470-b06b-49c5-9220-2db08ecfde86-kube-api-access-lbqbh\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:56:14.831341 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.831317    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zstf\" (UniqueName: \"kubernetes.io/projected/53ea0d03-1f13-4ee1-af04-9ddf449d6b33-kube-api-access-6zstf\") pod \"console-operator-7cd56b8dbc-2qq64\" (UID: \"53ea0d03-1f13-4ee1-af04-9ddf449d6b33\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:14.831501 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.831482    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgndt\" (UniqueName: \"kubernetes.io/projected/81050700-fe2d-45db-b66e-c83400cb2d35-kube-api-access-xgndt\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:14.831542 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.831482    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqzcg\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-kube-api-access-vqzcg\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:14.831542 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.831535    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v4hm\" (UniqueName: \"kubernetes.io/projected/6384f50d-3bbe-47ef-8e56-5a72d8113c6f-kube-api-access-8v4hm\") pod \"service-ca-operator-544d749c8c-9mfg4\" (UID: \"6384f50d-3bbe-47ef-8e56-5a72d8113c6f\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4"
Nov 02 11:56:14.831696 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.831681    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xjgq\" (UniqueName: \"kubernetes.io/projected/98301ff8-eb60-43d4-857f-da7d8da3858b-kube-api-access-2xjgq\") pod \"network-check-source-6f78d59f99-g7gnd\" (UID: \"98301ff8-eb60-43d4-857f-da7d8da3858b\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-g7gnd"
Nov 02 11:56:14.835683 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.835663    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lj85s\" (UniqueName: \"kubernetes.io/projected/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-api-access-lj85s\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:14.835762 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.835746    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g8ls\" (UniqueName: \"kubernetes.io/projected/e81ecb7e-0219-4976-b881-8c87ab61db50-kube-api-access-7g8ls\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.835958 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.835943    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbnkr\" (UniqueName: \"kubernetes.io/projected/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-kube-api-access-lbnkr\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.867905 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.867889    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-var-lib-cni-bin\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.867993 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.867913    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-modprobe-d\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.867993 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.867930    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-systemd\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.867993 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.867944    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:14.867993 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.867958    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-run-k8s-cni-cncf-io\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.867993 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.867987    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-cni-netd\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868003    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-run-netns\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868024    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-node-log\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868025    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-systemd\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868039    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-var-lib-cni-bin\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868073    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-run-netns\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868079    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-node-log\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868087    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-var-lib-cni-multus\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868056    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-cni-netd\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868039    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-run-k8s-cni-cncf-io\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868072    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-modprobe-d\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868049    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-var-lib-cni-multus\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868130    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-etc-kubernetes\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.868060    4942 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868144    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-sys\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868159    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-dbus\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868174    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-host\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868182    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-etc-kubernetes\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.868212    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:15.368197457 +0000 UTC m=+1.939547205 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : secret "router-metrics-certs-default" not found
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868219    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-sys\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868190    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a093470-b06b-49c5-9220-2db08ecfde86-system-cni-dir\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:56:14.868227 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868232    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a093470-b06b-49c5-9220-2db08ecfde86-system-cni-dir\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868246    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-host\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868257    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a093470-b06b-49c5-9220-2db08ecfde86-cnibin\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868282    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-multus-conf-dir\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868262    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-dbus\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868314    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-run-systemd\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868339    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a093470-b06b-49c5-9220-2db08ecfde86-cnibin\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868344    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-multus-conf-dir\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868348    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868353    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-run-systemd\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868379    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868395    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-kubernetes\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868410    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-run\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.868427    4942 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868430    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a093470-b06b-49c5-9220-2db08ecfde86-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868447    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-kubernetes\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868458    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-var-lib-kubelet\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868465    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-run\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.868482    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:15.368470066 +0000 UTC m=+1.939819817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : configmap references non-existent config key: service-ca.crt
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868484    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a093470-b06b-49c5-9220-2db08ecfde86-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mwslp\" (UID: \"0a093470-b06b-49c5-9220-2db08ecfde86\") " pod="openshift-multus/multus-additional-cni-plugins-mwslp"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868498    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-run-ovn\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868504    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-var-lib-kubelet\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.868508    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls podName:81050700-fe2d-45db-b66e-c83400cb2d35 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:15.368494243 +0000 UTC m=+1.939843992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2dm77" (UID: "81050700-fe2d-45db-b66e-c83400cb2d35") : secret "cluster-monitoring-operator-tls" not found
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868524    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-run-ovn\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868541    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868558    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-device-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868572    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-sys-fs\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868588    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-multus-cni-dir\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868603    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-cni-bin\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868627    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868636    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-device-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868646    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868651    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-system-cni-dir\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868659    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-cni-bin\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868660    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-multus-cni-dir\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868685    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-sysconfig\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868701    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-sys-fs\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.868705    4942 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868711    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-system-cni-dir\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868710    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-log-socket\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868730    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-sysconfig\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.868746    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs podName:60dc0a20-c792-4343-aec8-7ff8f86955e9 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:15.368735441 +0000 UTC m=+1.940085209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs") pod "network-metrics-daemon-n4tth" (UID: "60dc0a20-c792-4343-aec8-7ff8f86955e9") : secret "metrics-daemon-secret" not found
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868744    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-log-socket\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868762    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-run-ovn-kubernetes\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868788    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-run-multus-certs\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868790    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-run-ovn-kubernetes\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868810    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-cnibin\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868829    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-kubelet\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868831    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-run-multus-certs\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868843    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-registration-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868865    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-hostroot\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868891    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-sysctl-conf\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868903    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-hostroot\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868905    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-kubelet\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868913    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-cnibin\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868915    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-etc-selinux\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868943    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-registration-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868952    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868973    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-etc-openvswitch\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.868987    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc644b68-76cd-4326-8a15-2f8e3dd393d0-host\") pod \"node-ca-xt7dv\" (UID: \"fc644b68-76cd-4326-8a15-2f8e3dd393d0\") " pod="openshift-image-registry/node-ca-xt7dv"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869011    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-sysctl-d\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869019    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-etc-openvswitch\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869025    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc644b68-76cd-4326-8a15-2f8e3dd393d0-host\") pod \"node-ca-xt7dv\" (UID: \"fc644b68-76cd-4326-8a15-2f8e3dd393d0\") " pod="openshift-image-registry/node-ca-xt7dv"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869033    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869040    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869021    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-etc-selinux\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869022    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-sysctl-conf\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869064    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-kubelet-config\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869101    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-lib-modules\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.869114    4942 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 11:56:14.869068 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.869127    4942 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75766bbfd6-gpgxx: secret "image-registry-tls" not found
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869129    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6d8ba8e4-34a6-47eb-a33f-b67c52acd079-kubelet-config\") pod \"global-pull-secret-syncer-vm62x\" (UID: \"6d8ba8e4-34a6-47eb-a33f-b67c52acd079\") " pod="kube-system/global-pull-secret-syncer-vm62x"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869128    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/252df0cc-a5c0-4dea-aca5-bd6b08262d38-crio-socket\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869113    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-etc-sysctl-d\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869173    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ca3e4d6-0eac-4309-b049-e5f597728119-host-slash\") pod \"iptables-alerter-dqg86\" (UID: \"5ca3e4d6-0eac-4309-b049-e5f597728119\") " pod="openshift-network-operator/iptables-alerter-dqg86"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869179    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/252df0cc-a5c0-4dea-aca5-bd6b08262d38-crio-socket\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869194    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ca3e4d6-0eac-4309-b049-e5f597728119-host-slash\") pod \"iptables-alerter-dqg86\" (UID: \"5ca3e4d6-0eac-4309-b049-e5f597728119\") " pod="openshift-network-operator/iptables-alerter-dqg86"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.869205    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls podName:2759fb5a-17ad-4c29-b9c8-175769f53069 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:15.36919059 +0000 UTC m=+1.940540339 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls") pod "image-registry-75766bbfd6-gpgxx" (UID: "2759fb5a-17ad-4c29-b9c8-175769f53069") : secret "image-registry-tls" not found
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869227    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-lib-modules\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869227    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-multus-socket-dir-parent\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869264    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-systemd-units\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869272    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-multus-socket-dir-parent\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869280    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-var-lib-openvswitch\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869312    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869326    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-var-lib-openvswitch\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869332    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d239eb0-492f-4335-857e-4edc754823e4-hosts-file\") pod \"node-resolver-h24nt\" (UID: \"4d239eb0-492f-4335-857e-4edc754823e4\") " pod="openshift-dns/node-resolver-h24nt"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869331    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-systemd-units\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869363    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d239eb0-492f-4335-857e-4edc754823e4-hosts-file\") pod \"node-resolver-h24nt\" (UID: \"4d239eb0-492f-4335-857e-4edc754823e4\") " pod="openshift-dns/node-resolver-h24nt"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869380    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-socket-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.869397    4942 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.869411    4942 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d58f4b759-ts6c2: secret "image-registry-tls" not found
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869397    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.869447    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls podName:e9dc2cae-7e99-4282-a02d-51fb0f12b43b nodeName:}" failed. No retries permitted until 2026-11-02 11:56:15.369435773 +0000 UTC m=+1.940785525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls") pod "image-registry-6d58f4b759-ts6c2" (UID: "e9dc2cae-7e99-4282-a02d-51fb0f12b43b") : secret "image-registry-tls" not found
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869449    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6ee9fb0f-d08d-4e09-87bf-df21a5309ca4-socket-dir\") pod \"aws-ebs-csi-driver-node-vrkfk\" (UID: \"6ee9fb0f-d08d-4e09-87bf-df21a5309ca4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vrkfk"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869475    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-slash\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869493    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-run-openvswitch\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869512    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869542    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869545    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-host-slash\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869559    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e81ecb7e-0219-4976-b881-8c87ab61db50-run-openvswitch\") pod \"ovnkube-node-7686t\" (UID: \"e81ecb7e-0219-4976-b881-8c87ab61db50\") " pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869570    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-run-netns\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.869596    4942 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.869614    4942 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.869636    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls podName:252df0cc-a5c0-4dea-aca5-bd6b08262d38 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:15.369621659 +0000 UTC m=+1.940971421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-krn29" (UID: "252df0cc-a5c0-4dea-aca5-bd6b08262d38") : secret "insights-runtime-extractor-tls" not found
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869638    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/68728c63-ace5-4888-a19b-2782fd2ba673-host-run-netns\") pod \"multus-fsq7c\" (UID: \"68728c63-ace5-4888-a19b-2782fd2ba673\") " pod="openshift-multus/multus-fsq7c"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869594    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.869653    4942 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.869662    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls podName:bbfab1b5-14db-477a-8f06-7cf2c1d058a2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:15.369648601 +0000 UTC m=+1.940998350 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls") pod "dns-default-cttl7" (UID: "bbfab1b5-14db-477a-8f06-7cf2c1d058a2") : secret "dns-default-metrics-tls" not found
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.869698    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls podName:f38c9faa-5fae-45ba-93f6-e252b6f092fd nodeName:}" failed. No retries permitted until 2026-11-02 11:56:15.369686852 +0000 UTC m=+1.941036602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-9vksm" (UID: "f38c9faa-5fae-45ba-93f6-e252b6f092fd") : secret "samples-operator-tls" not found
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869712    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869739    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-var-lib-kubelet\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.869768    4942 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869789    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2dc02c96-a4d9-46ab-8b52-5382cf43b92c-var-lib-kubelet\") pod \"tuned-sv47s\" (UID: \"2dc02c96-a4d9-46ab-8b52-5382cf43b92c\") " pod="openshift-cluster-node-tuning-operator/tuned-sv47s"
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:14.869810    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert podName:c0e3ecc0-e47b-4612-87aa-53ce8434f5a8 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:15.36979848 +0000 UTC m=+1.941148234 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-hflkz" (UID: "c0e3ecc0-e47b-4612-87aa-53ce8434f5a8") : secret "networking-console-plugin-cert" not found
Nov 02 11:56:14.871239 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.869811    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/252df0cc-a5c0-4dea-aca5-bd6b08262d38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:14.946136 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:14.946113    4942 scope.go:122] "RemoveContainer" containerID="c47445652e8495da228c46a9bb8c8775699f2a05b984ce730a61e30adc8d8cfd"
Nov 02 11:56:15.374582 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:15.374554    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:15.374750 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:15.374590    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:15.374750 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:15.374616    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:56:15.374750 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.374705    4942 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 11:56:15.374750 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:15.374731    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:15.374750 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.374747    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs podName:60dc0a20-c792-4343-aec8-7ff8f86955e9 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:16.37473542 +0000 UTC m=+2.946085155 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs") pod "network-metrics-daemon-n4tth" (UID: "60dc0a20-c792-4343-aec8-7ff8f86955e9") : secret "metrics-daemon-secret" not found
Nov 02 11:56:15.374927 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.374707    4942 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 11:56:15.374927 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.374790    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:16.374774768 +0000 UTC m=+2.946124515 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : configmap references non-existent config key: service-ca.crt
Nov 02 11:56:15.374927 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.374792    4942 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 11:56:15.374927 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.374806    4942 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75766bbfd6-gpgxx: secret "image-registry-tls" not found
Nov 02 11:56:15.374927 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:15.374808    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:15.374927 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.374821    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls podName:81050700-fe2d-45db-b66e-c83400cb2d35 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:16.374808675 +0000 UTC m=+2.946158411 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2dm77" (UID: "81050700-fe2d-45db-b66e-c83400cb2d35") : secret "cluster-monitoring-operator-tls" not found
Nov 02 11:56:15.374927 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.374855    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls podName:2759fb5a-17ad-4c29-b9c8-175769f53069 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:16.374841826 +0000 UTC m=+2.946191575 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls") pod "image-registry-75766bbfd6-gpgxx" (UID: "2759fb5a-17ad-4c29-b9c8-175769f53069") : secret "image-registry-tls" not found
Nov 02 11:56:15.374927 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.374861    4942 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 11:56:15.374927 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.374871    4942 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d58f4b759-ts6c2: secret "image-registry-tls" not found
Nov 02 11:56:15.374927 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:15.374890    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:15.374927 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.374895    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls podName:e9dc2cae-7e99-4282-a02d-51fb0f12b43b nodeName:}" failed. No retries permitted until 2026-11-02 11:56:16.374887739 +0000 UTC m=+2.946237475 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls") pod "image-registry-6d58f4b759-ts6c2" (UID: "e9dc2cae-7e99-4282-a02d-51fb0f12b43b") : secret "image-registry-tls" not found
Nov 02 11:56:15.374927 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:15.374919    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:15.374927 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.374934    4942 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 11:56:15.375340 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:15.374936    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:15.375340 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.374957    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls podName:252df0cc-a5c0-4dea-aca5-bd6b08262d38 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:16.374947918 +0000 UTC m=+2.946297652 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-krn29" (UID: "252df0cc-a5c0-4dea-aca5-bd6b08262d38") : secret "insights-runtime-extractor-tls" not found
Nov 02 11:56:15.375340 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.374979    4942 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 11:56:15.375340 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.375000    4942 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 11:56:15.375340 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.375009    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls podName:bbfab1b5-14db-477a-8f06-7cf2c1d058a2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:16.375001737 +0000 UTC m=+2.946351472 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls") pod "dns-default-cttl7" (UID: "bbfab1b5-14db-477a-8f06-7cf2c1d058a2") : secret "dns-default-metrics-tls" not found
Nov 02 11:56:15.375340 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.375010    4942 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 11:56:15.375340 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:15.374975    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:15.375340 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.375027    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls podName:f38c9faa-5fae-45ba-93f6-e252b6f092fd nodeName:}" failed. No retries permitted until 2026-11-02 11:56:16.375019438 +0000 UTC m=+2.946369207 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-9vksm" (UID: "f38c9faa-5fae-45ba-93f6-e252b6f092fd") : secret "samples-operator-tls" not found
Nov 02 11:56:15.375340 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.375045    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert podName:c0e3ecc0-e47b-4612-87aa-53ce8434f5a8 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:16.375037588 +0000 UTC m=+2.946387326 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-hflkz" (UID: "c0e3ecc0-e47b-4612-87aa-53ce8434f5a8") : secret "networking-console-plugin-cert" not found
Nov 02 11:56:15.375340 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:15.375071    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:15.375340 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.375151    4942 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 11:56:15.375340 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.375188    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:16.375180492 +0000 UTC m=+2.946530232 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : secret "router-metrics-certs-default" not found
Nov 02 11:56:15.591873 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:15.591843    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 11:56:15.592278 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:15.592262    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/1.log"
Nov 02 11:56:15.592379 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:15.592306    4942 generic.go:356] "Generic (PLEG): container finished" podID="53ea0d03-1f13-4ee1-af04-9ddf449d6b33" containerID="dfa564d158c74c542a934776bfb99ef5db43e10afde1dfdc98855063bbae53d3" exitCode=255
Nov 02 11:56:15.592379 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:15.592324    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64" event={"ID":"53ea0d03-1f13-4ee1-af04-9ddf449d6b33","Type":"ContainerDied","Data":"dfa564d158c74c542a934776bfb99ef5db43e10afde1dfdc98855063bbae53d3"}
Nov 02 11:56:15.592452 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:15.592385    4942 scope.go:122] "RemoveContainer" containerID="c47445652e8495da228c46a9bb8c8775699f2a05b984ce730a61e30adc8d8cfd"
Nov 02 11:56:15.592952 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:15.592933    4942 scope.go:122] "RemoveContainer" containerID="dfa564d158c74c542a934776bfb99ef5db43e10afde1dfdc98855063bbae53d3"
Nov 02 11:56:15.593085 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:15.593068    4942 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2qq64_openshift-console-operator(53ea0d03-1f13-4ee1-af04-9ddf449d6b33)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64" podUID="53ea0d03-1f13-4ee1-af04-9ddf449d6b33"
Nov 02 11:56:16.384883 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:16.384853    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:16.384883 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:16.384888    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:16.385077 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:16.384906    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:16.385077 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:16.384935    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:16.385077 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:16.384987    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:16.385077 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.384999    4942 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 11:56:16.385077 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385012    4942 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 11:56:16.385077 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385054    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls podName:bbfab1b5-14db-477a-8f06-7cf2c1d058a2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:18.385036204 +0000 UTC m=+4.956385952 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls") pod "dns-default-cttl7" (UID: "bbfab1b5-14db-477a-8f06-7cf2c1d058a2") : secret "dns-default-metrics-tls" not found
Nov 02 11:56:16.385077 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:16.385011    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:16.385077 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385066    4942 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 11:56:16.385378 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385089    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:18.385071594 +0000 UTC m=+4.956421351 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : configmap references non-existent config key: service-ca.crt
Nov 02 11:56:16.385378 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385091    4942 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 11:56:16.385378 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385110    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls podName:f38c9faa-5fae-45ba-93f6-e252b6f092fd nodeName:}" failed. No retries permitted until 2026-11-02 11:56:18.385101499 +0000 UTC m=+4.956451247 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-9vksm" (UID: "f38c9faa-5fae-45ba-93f6-e252b6f092fd") : secret "samples-operator-tls" not found
Nov 02 11:56:16.385378 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385133    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:18.385118631 +0000 UTC m=+4.956468383 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : secret "router-metrics-certs-default" not found
Nov 02 11:56:16.385378 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385012    4942 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 11:56:16.385378 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385163    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls podName:81050700-fe2d-45db-b66e-c83400cb2d35 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:18.385152122 +0000 UTC m=+4.956501871 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2dm77" (UID: "81050700-fe2d-45db-b66e-c83400cb2d35") : secret "cluster-monitoring-operator-tls" not found
Nov 02 11:56:16.385378 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:16.385196    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:56:16.385378 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385213    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert podName:c0e3ecc0-e47b-4612-87aa-53ce8434f5a8 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:18.385200346 +0000 UTC m=+4.956550085 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-hflkz" (UID: "c0e3ecc0-e47b-4612-87aa-53ce8434f5a8") : secret "networking-console-plugin-cert" not found
Nov 02 11:56:16.385378 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385243    4942 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 11:56:16.385378 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385265    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs podName:60dc0a20-c792-4343-aec8-7ff8f86955e9 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:18.385258596 +0000 UTC m=+4.956608373 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs") pod "network-metrics-daemon-n4tth" (UID: "60dc0a20-c792-4343-aec8-7ff8f86955e9") : secret "metrics-daemon-secret" not found
Nov 02 11:56:16.385378 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:16.385264    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:16.385378 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:16.385290    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:16.385378 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:16.385334    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:16.385767 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385399    4942 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 11:56:16.385767 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385408    4942 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d58f4b759-ts6c2: secret "image-registry-tls" not found
Nov 02 11:56:16.385767 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385410    4942 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 02 11:56:16.385767 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385421    4942 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75766bbfd6-gpgxx: secret "image-registry-tls" not found
Nov 02 11:56:16.385767 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385438    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls podName:e9dc2cae-7e99-4282-a02d-51fb0f12b43b nodeName:}" failed. No retries permitted until 2026-11-02 11:56:18.385431059 +0000 UTC m=+4.956780794 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls") pod "image-registry-6d58f4b759-ts6c2" (UID: "e9dc2cae-7e99-4282-a02d-51fb0f12b43b") : secret "image-registry-tls" not found
Nov 02 11:56:16.385767 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385411    4942 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 11:56:16.385767 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385456    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls podName:2759fb5a-17ad-4c29-b9c8-175769f53069 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:18.385447902 +0000 UTC m=+4.956797678 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls") pod "image-registry-75766bbfd6-gpgxx" (UID: "2759fb5a-17ad-4c29-b9c8-175769f53069") : secret "image-registry-tls" not found
Nov 02 11:56:16.385767 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:16.385466    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls podName:252df0cc-a5c0-4dea-aca5-bd6b08262d38 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:18.38546119 +0000 UTC m=+4.956810925 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-krn29" (UID: "252df0cc-a5c0-4dea-aca5-bd6b08262d38") : secret "insights-runtime-extractor-tls" not found
Nov 02 11:56:16.597044 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:16.597017    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 11:56:16.631771 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:16.631748    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-797nn"
Nov 02 11:56:16.903502 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:16.903472    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:56:17.803570 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:17.803547    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:18.402420 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:18.402369    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:18.402420 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:18.402405    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:18.402675 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:18.402443    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:56:18.402675 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:18.402478    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:18.402675 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:18.402498    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:18.402675 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:18.402526    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:18.402675 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:18.402560    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:22.402541476 +0000 UTC m=+8.973891227 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : configmap references non-existent config key: service-ca.crt
Nov 02 11:56:18.402675 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:18.402590    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:18.402675 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:18.402597    4942 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 02 11:56:18.402675 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:18.402644    4942 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 02 11:56:18.402675 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:18.402597    4942 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 11:56:18.403092 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:18.402686    4942 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 11:56:18.403092 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:18.402627    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:18.403092 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:18.402688    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls podName:bbfab1b5-14db-477a-8f06-7cf2c1d058a2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:22.40267389 +0000 UTC m=+8.974023629 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls") pod "dns-default-cttl7" (UID: "bbfab1b5-14db-477a-8f06-7cf2c1d058a2") : secret "dns-default-metrics-tls" not found
Nov 02 11:56:18.403092 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:18.402597    4942 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 02 11:56:18.403092 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:18.402742    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls podName:81050700-fe2d-45db-b66e-c83400cb2d35 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:22.402729525 +0000 UTC m=+8.974079267 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2dm77" (UID: "81050700-fe2d-45db-b66e-c83400cb2d35") : secret "cluster-monitoring-operator-tls" not found
Nov 02 11:56:18.403092 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:18.402773    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls podName:f38c9faa-5fae-45ba-93f6-e252b6f092fd nodeName:}" failed. No retries permitted until 2026-11-02 11:56:22.402762269 +0000 UTC m=+8.974112039 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-9vksm" (UID: "f38c9faa-5fae-45ba-93f6-e252b6f092fd") : secret "samples-operator-tls" not found
Nov 02 11:56:18.403092 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:18.402789    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls podName:252df0cc-a5c0-4dea-aca5-bd6b08262d38 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:22.402780333 +0000 UTC m=+8.974130074 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-krn29" (UID: "252df0cc-a5c0-4dea-aca5-bd6b08262d38") : secret "insights-runtime-extractor-tls" not found
Nov 02 11:56:18.403092 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:18.402804    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs podName:60dc0a20-c792-4343-aec8-7ff8f86955e9 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:22.402795317 +0000 UTC m=+8.974145052 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs") pod "network-metrics-daemon-n4tth" (UID: "60dc0a20-c792-4343-aec8-7ff8f86955e9") : secret "metrics-daemon-secret" not found
Nov 02 11:56:18.403092 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:18.402822    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:18.403092 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:18.402862    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:18.403092 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:18.402864    4942 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 02 11:56:18.403092 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:18.402910    4942 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 11:56:18.403092 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:18.402949    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert podName:c0e3ecc0-e47b-4612-87aa-53ce8434f5a8 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:22.402939688 +0000 UTC m=+8.974289441 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-hflkz" (UID: "c0e3ecc0-e47b-4612-87aa-53ce8434f5a8") : secret "networking-console-plugin-cert" not found
Nov 02 11:56:18.403092 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:18.402974    4942 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 02 11:56:18.403092 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:18.403011    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:22.402999234 +0000 UTC m=+8.974348986 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : secret "router-metrics-certs-default" not found
Nov 02 11:56:18.405195 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:18.405167    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls\") pod \"image-registry-75766bbfd6-gpgxx\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") " pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:18.405309 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:18.405178    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls\") pod \"image-registry-6d58f4b759-ts6c2\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") " pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:18.545209 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:18.545184    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:18.555121 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:18.555102    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:19.020888 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:19.020854    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75766bbfd6-gpgxx"]
Nov 02 11:56:19.024072 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:19.024050    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d58f4b759-ts6c2"]
Nov 02 11:56:19.057566 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:19.057540    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2759fb5a_17ad_4c29_b9c8_175769f53069.slice/crio-1691d6cc3707d6adf663623b810246ea6f41ccde3d2b1a8a477e173e507aaf4c WatchSource:0}: Error finding container 1691d6cc3707d6adf663623b810246ea6f41ccde3d2b1a8a477e173e507aaf4c: Status 404 returned error can't find the container with id 1691d6cc3707d6adf663623b810246ea6f41ccde3d2b1a8a477e173e507aaf4c
Nov 02 11:56:19.063957 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:19.063934    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:19.064249 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:19.064235    4942 scope.go:122] "RemoveContainer" containerID="dfa564d158c74c542a934776bfb99ef5db43e10afde1dfdc98855063bbae53d3"
Nov 02 11:56:19.064404 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:19.064385    4942 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2qq64_openshift-console-operator(53ea0d03-1f13-4ee1-af04-9ddf449d6b33)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64" podUID="53ea0d03-1f13-4ee1-af04-9ddf449d6b33"
Nov 02 11:56:19.071797 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:19.071775    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-h24nt_4d239eb0-492f-4335-857e-4edc754823e4/dns-node-resolver/0.log"
Nov 02 11:56:19.073369 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:19.073340    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9dc2cae_7e99_4282_a02d_51fb0f12b43b.slice/crio-3451f27745f1ab8a2b397a8f00ac679a44a3948c2bd9206528ec990edf6eee33 WatchSource:0}: Error finding container 3451f27745f1ab8a2b397a8f00ac679a44a3948c2bd9206528ec990edf6eee33: Status 404 returned error can't find the container with id 3451f27745f1ab8a2b397a8f00ac679a44a3948c2bd9206528ec990edf6eee33
Nov 02 11:56:19.612264 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:19.612230    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx" event={"ID":"2759fb5a-17ad-4c29-b9c8-175769f53069","Type":"ContainerStarted","Data":"51cd52accb53dd63a7aa5d4e18d45d0e007076c89f32a1db782ed015b71ce0ff"}
Nov 02 11:56:19.612481 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:19.612286    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx" event={"ID":"2759fb5a-17ad-4c29-b9c8-175769f53069","Type":"ContainerStarted","Data":"1691d6cc3707d6adf663623b810246ea6f41ccde3d2b1a8a477e173e507aaf4c"}
Nov 02 11:56:19.613814 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:19.613779    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2" event={"ID":"e9dc2cae-7e99-4282-a02d-51fb0f12b43b","Type":"ContainerStarted","Data":"c21c939fbd73af2997869fe4e2d17bb78541c62b87ca50a0af6bc652d6d0cee2"}
Nov 02 11:56:19.613935 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:19.613818    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2" event={"ID":"e9dc2cae-7e99-4282-a02d-51fb0f12b43b","Type":"ContainerStarted","Data":"3451f27745f1ab8a2b397a8f00ac679a44a3948c2bd9206528ec990edf6eee33"}
Nov 02 11:56:19.671400 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:19.671377    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-75766bbfd6-gpgxx_2759fb5a-17ad-4c29-b9c8-175769f53069/registry/0.log"
Nov 02 11:56:20.271889 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:20.271865    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xt7dv_fc644b68-76cd-4326-8a15-2f8e3dd393d0/node-ca/0.log"
Nov 02 11:56:20.616554 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:20.616529    4942 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 11:56:20.616712 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:20.616528    4942 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 11:56:20.629443 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:20.629421    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:20.629715 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:20.629698    4942 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 02 11:56:20.872272 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:20.872201    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tlvw4_13d85b4b-e889-48b8-97af-f23632533bea/serve-healthcheck-canary/0.log"
Nov 02 11:56:21.272931 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:21.272858    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-gdvmz_6238f603-d9da-4f8a-98a8-31e69a11b625/migrator/0.log"
Nov 02 11:56:21.473875 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:21.473851    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf_691a1fbe-dd29-4579-8e72-8c36c2b1eda2/kube-storage-version-migrator-operator/0.log"
Nov 02 11:56:22.442380 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:22.442340    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:22.442769 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:22.442389    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:22.442769 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:22.442423    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:22.442769 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:22.442440    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:22.442769 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:22.442473    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:56:22.442769 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:22.442482    4942 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 02 11:56:22.442769 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:22.442534    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:22.442769 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:22.442573    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert podName:c0e3ecc0-e47b-4612-87aa-53ce8434f5a8 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:30.44255329 +0000 UTC m=+17.013903025 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-hflkz" (UID: "c0e3ecc0-e47b-4612-87aa-53ce8434f5a8") : secret "networking-console-plugin-cert" not found
Nov 02 11:56:22.442769 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:22.442606    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle podName:15291745-62ec-45f2-85f9-373504d79de2 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:30.442591955 +0000 UTC m=+17.013941706 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle") pod "router-default-567988bb57-rzsqv" (UID: "15291745-62ec-45f2-85f9-373504d79de2") : configmap references non-existent config key: service-ca.crt
Nov 02 11:56:22.442769 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:22.442601    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:22.442769 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:22.442616    4942 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 02 11:56:22.442769 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:22.442647    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:22.442769 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:22.442678    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls podName:252df0cc-a5c0-4dea-aca5-bd6b08262d38 nodeName:}" failed. No retries permitted until 2026-11-02 11:56:30.442662331 +0000 UTC m=+17.014012068 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-krn29" (UID: "252df0cc-a5c0-4dea-aca5-bd6b08262d38") : secret "insights-runtime-extractor-tls" not found
Nov 02 11:56:22.442769 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:22.442740    4942 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 02 11:56:22.443313 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:22.442785    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls podName:f38c9faa-5fae-45ba-93f6-e252b6f092fd nodeName:}" failed. No retries permitted until 2026-11-02 11:56:30.442774051 +0000 UTC m=+17.014123786 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-9vksm" (UID: "f38c9faa-5fae-45ba-93f6-e252b6f092fd") : secret "samples-operator-tls" not found
Nov 02 11:56:22.444833 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:22.444809    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60dc0a20-c792-4343-aec8-7ff8f86955e9-metrics-certs\") pod \"network-metrics-daemon-n4tth\" (UID: \"60dc0a20-c792-4343-aec8-7ff8f86955e9\") " pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:56:22.445074 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:22.445048    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/81050700-fe2d-45db-b66e-c83400cb2d35-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dm77\" (UID: \"81050700-fe2d-45db-b66e-c83400cb2d35\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:22.445132 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:22.445118    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbfab1b5-14db-477a-8f06-7cf2c1d058a2-metrics-tls\") pod \"dns-default-cttl7\" (UID: \"bbfab1b5-14db-477a-8f06-7cf2c1d058a2\") " pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:22.445184 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:22.445164    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15291745-62ec-45f2-85f9-373504d79de2-metrics-certs\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:22.449106 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:22.449087    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n4tth"
Nov 02 11:56:22.456697 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:22.456668    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:22.744107 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:22.744035    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"
Nov 02 11:56:22.817058 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:22.817032    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n4tth"]
Nov 02 11:56:22.819830 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:22.819806    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60dc0a20_c792_4343_aec8_7ff8f86955e9.slice/crio-b0b3d9614435c606a52a03d7f2d929b23c51a26bc0d31a267cdaf0c60e06bf79 WatchSource:0}: Error finding container b0b3d9614435c606a52a03d7f2d929b23c51a26bc0d31a267cdaf0c60e06bf79: Status 404 returned error can't find the container with id b0b3d9614435c606a52a03d7f2d929b23c51a26bc0d31a267cdaf0c60e06bf79
Nov 02 11:56:22.821909 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:22.821894    4942 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 11:56:22.868539 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:22.868519    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cttl7"]
Nov 02 11:56:22.873384 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:22.873356    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbfab1b5_14db_477a_8f06_7cf2c1d058a2.slice/crio-a4d2c525beb9b8eb349550be7e3b7e3d414542b5b716239f4497006df2c74be5 WatchSource:0}: Error finding container a4d2c525beb9b8eb349550be7e3b7e3d414542b5b716239f4497006df2c74be5: Status 404 returned error can't find the container with id a4d2c525beb9b8eb349550be7e3b7e3d414542b5b716239f4497006df2c74be5
Nov 02 11:56:23.108351 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:23.108325    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77"]
Nov 02 11:56:23.111862 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:23.111839    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81050700_fe2d_45db_b66e_c83400cb2d35.slice/crio-8b893ca5440e9e63b79fd0fe8e4c2dda580509bae32221dc9cfa37c815634655 WatchSource:0}: Error finding container 8b893ca5440e9e63b79fd0fe8e4c2dda580509bae32221dc9cfa37c815634655: Status 404 returned error can't find the container with id 8b893ca5440e9e63b79fd0fe8e4c2dda580509bae32221dc9cfa37c815634655
Nov 02 11:56:23.414115 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:23.414040    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:23.626911 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:23.626879    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cttl7" event={"ID":"bbfab1b5-14db-477a-8f06-7cf2c1d058a2","Type":"ContainerStarted","Data":"a4d2c525beb9b8eb349550be7e3b7e3d414542b5b716239f4497006df2c74be5"}
Nov 02 11:56:23.628048 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:23.628021    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77" event={"ID":"81050700-fe2d-45db-b66e-c83400cb2d35","Type":"ContainerStarted","Data":"8b893ca5440e9e63b79fd0fe8e4c2dda580509bae32221dc9cfa37c815634655"}
Nov 02 11:56:23.629067 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:23.629047    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n4tth" event={"ID":"60dc0a20-c792-4343-aec8-7ff8f86955e9","Type":"ContainerStarted","Data":"b0b3d9614435c606a52a03d7f2d929b23c51a26bc0d31a267cdaf0c60e06bf79"}
Nov 02 11:56:24.101365 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:24.101323    4942 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-797nn_6b350fc9-6bb7-43c4-9b89-c5157f42ca26/konnectivity-agent/0.log" limit=1024
Nov 02 11:56:24.416719 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:24.415940    4942 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:24.416719 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:24.416471    4942 scope.go:122] "RemoveContainer" containerID="dfa564d158c74c542a934776bfb99ef5db43e10afde1dfdc98855063bbae53d3"
Nov 02 11:56:24.416719 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:24.416664    4942 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2qq64_openshift-console-operator(53ea0d03-1f13-4ee1-af04-9ddf449d6b33)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64" podUID="53ea0d03-1f13-4ee1-af04-9ddf449d6b33"
Nov 02 11:56:24.634856 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:24.634777    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n4tth" event={"ID":"60dc0a20-c792-4343-aec8-7ff8f86955e9","Type":"ContainerStarted","Data":"4e43f02b48cef0d80633470d6a4474171b0b5af723df2395886fe5ddcff6e95b"}
Nov 02 11:56:24.634856 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:24.634843    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n4tth" event={"ID":"60dc0a20-c792-4343-aec8-7ff8f86955e9","Type":"ContainerStarted","Data":"f7ee60f0a92e544fe93057363917775d40faac92fde303a6821ed503ed3f85f0"}
Nov 02 11:56:25.530045 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:25.530016    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:25.535786 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:25.535767    4942 scope.go:122] "RemoveContainer" containerID="dfa564d158c74c542a934776bfb99ef5db43e10afde1dfdc98855063bbae53d3"
Nov 02 11:56:25.639936 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:25.639906    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cttl7" event={"ID":"bbfab1b5-14db-477a-8f06-7cf2c1d058a2","Type":"ContainerStarted","Data":"8075bbb36a2b538fbb40aa8118825bf7bd5a8a0d7836e6f6fd225a45ed5dbe9a"}
Nov 02 11:56:25.640394 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:25.639956    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cttl7" event={"ID":"bbfab1b5-14db-477a-8f06-7cf2c1d058a2","Type":"ContainerStarted","Data":"518270304a967a09381281654b44c8a1a92a9b803ef73406e8f1cf80fa496388"}
Nov 02 11:56:25.640394 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:25.639996    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:25.641627 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:25.641599    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dm77" event={"ID":"81050700-fe2d-45db-b66e-c83400cb2d35","Type":"ContainerStarted","Data":"c1a93c8a9ddf16e2866ecc06924686747e9472ddac82ba8d09723a2cd4886608"}
Nov 02 11:56:25.643859 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:25.643841    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 11:56:25.643956 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:25.643939    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64" event={"ID":"53ea0d03-1f13-4ee1-af04-9ddf449d6b33","Type":"ContainerStarted","Data":"a7457266a4f286f4c93241fbe1ecf0197155193667be313d1bc8334f05c223fb"}
Nov 02 11:56:25.644319 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:25.644274    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:25.645222 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:25.645198    4942 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-2qq64 container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.13:8443/readyz\": dial tcp 10.132.0.13:8443: connect: connection refused" start-of-body=
Nov 02 11:56:25.645340 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:25.645247    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64" podUID="53ea0d03-1f13-4ee1-af04-9ddf449d6b33" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.13:8443/readyz\": dial tcp 10.132.0.13:8443: connect: connection refused"
Nov 02 11:56:26.650786 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:26.650751    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-2qq64"
Nov 02 11:56:30.517667 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:30.517626    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:30.518050 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:30.517689    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:30.518050 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:30.517714    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:30.518050 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:30.517731    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:30.518366 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:30.518335    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15291745-62ec-45f2-85f9-373504d79de2-service-ca-bundle\") pod \"router-default-567988bb57-rzsqv\" (UID: \"15291745-62ec-45f2-85f9-373504d79de2\") " pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:30.520022 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:30.519988    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/252df0cc-a5c0-4dea-aca5-bd6b08262d38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-krn29\" (UID: \"252df0cc-a5c0-4dea-aca5-bd6b08262d38\") " pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:30.520138 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:30.520116    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f38c9faa-5fae-45ba-93f6-e252b6f092fd-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-9vksm\" (UID: \"f38c9faa-5fae-45ba-93f6-e252b6f092fd\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:30.520211 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:30.520184    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c0e3ecc0-e47b-4612-87aa-53ce8434f5a8-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hflkz\" (UID: \"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:30.544137 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:30.544115    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"
Nov 02 11:56:30.545220 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:30.545201    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:30.549053 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:30.549040    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz"
Nov 02 11:56:30.557464 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:30.557445    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-krn29"
Nov 02 11:56:31.054458 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:31.054168    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-hflkz"]
Nov 02 11:56:31.057775 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:31.057742    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0e3ecc0_e47b_4612_87aa_53ce8434f5a8.slice/crio-059d219a17b22678dd62c42621bb10d14c4eb9e71cbcd6b7c0d75f5f6e2d8d41 WatchSource:0}: Error finding container 059d219a17b22678dd62c42621bb10d14c4eb9e71cbcd6b7c0d75f5f6e2d8d41: Status 404 returned error can't find the container with id 059d219a17b22678dd62c42621bb10d14c4eb9e71cbcd6b7c0d75f5f6e2d8d41
Nov 02 11:56:31.103802 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:31.103781    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm"]
Nov 02 11:56:31.104434 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:31.104419    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-567988bb57-rzsqv"]
Nov 02 11:56:31.105064 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:31.105044    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-krn29"]
Nov 02 11:56:31.105579 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:31.105557    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15291745_62ec_45f2_85f9_373504d79de2.slice/crio-038500542d5866c158015ed667d4b77ce87beb7975205f5c7091db2362b58c33 WatchSource:0}: Error finding container 038500542d5866c158015ed667d4b77ce87beb7975205f5c7091db2362b58c33: Status 404 returned error can't find the container with id 038500542d5866c158015ed667d4b77ce87beb7975205f5c7091db2362b58c33
Nov 02 11:56:31.106232 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:31.106211    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod252df0cc_a5c0_4dea_aca5_bd6b08262d38.slice/crio-912d11e5f80a48cb2e1307bd4440869ac46ca1a4f041af702512bde7f7a6fcad WatchSource:0}: Error finding container 912d11e5f80a48cb2e1307bd4440869ac46ca1a4f041af702512bde7f7a6fcad: Status 404 returned error can't find the container with id 912d11e5f80a48cb2e1307bd4440869ac46ca1a4f041af702512bde7f7a6fcad
Nov 02 11:56:31.663950 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:31.663916    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-krn29" event={"ID":"252df0cc-a5c0-4dea-aca5-bd6b08262d38","Type":"ContainerStarted","Data":"1580649f2b139e86f570b601a80e2a8d4eac09046c7d4a24262715d2148563c6"}
Nov 02 11:56:31.664393 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:31.663984    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-krn29" event={"ID":"252df0cc-a5c0-4dea-aca5-bd6b08262d38","Type":"ContainerStarted","Data":"912d11e5f80a48cb2e1307bd4440869ac46ca1a4f041af702512bde7f7a6fcad"}
Nov 02 11:56:31.665233 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:31.665201    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz" event={"ID":"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8","Type":"ContainerStarted","Data":"059d219a17b22678dd62c42621bb10d14c4eb9e71cbcd6b7c0d75f5f6e2d8d41"}
Nov 02 11:56:31.666750 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:31.666724    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-567988bb57-rzsqv" event={"ID":"15291745-62ec-45f2-85f9-373504d79de2","Type":"ContainerStarted","Data":"3d7faa73e4f031b66ec498a95f85b376d7014bdd090b5dde6dfb5c62b9363d5a"}
Nov 02 11:56:31.666843 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:31.666765    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-567988bb57-rzsqv" event={"ID":"15291745-62ec-45f2-85f9-373504d79de2","Type":"ContainerStarted","Data":"038500542d5866c158015ed667d4b77ce87beb7975205f5c7091db2362b58c33"}
Nov 02 11:56:31.667940 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:31.667916    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm" event={"ID":"f38c9faa-5fae-45ba-93f6-e252b6f092fd","Type":"ContainerStarted","Data":"a908df6d62c89d8f65e3622622befc414a126067ddfe4b853ee7618691f03530"}
Nov 02 11:56:32.029177 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:32.029100    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75766bbfd6-gpgxx"]
Nov 02 11:56:32.546406 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:32.546361    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:32.549649 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:32.549622    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:32.674721 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:32.674682    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-krn29" event={"ID":"252df0cc-a5c0-4dea-aca5-bd6b08262d38","Type":"ContainerStarted","Data":"842d2af236c71b0669bdf845c1a6775e651ed8a9b6afe3aa06ad4dc338a0ec71"}
Nov 02 11:56:32.675121 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:32.674923    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:32.676684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:32.676587    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-567988bb57-rzsqv"
Nov 02 11:56:32.934037 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:32.933952    4942 patch_prober.go:29] interesting pod/image-registry-6d58f4b759-ts6c2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Nov 02 11:56:32.934037 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:32.934019    4942 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2" podUID="e9dc2cae-7e99-4282-a02d-51fb0f12b43b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 11:56:33.684615 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:33.684570    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-hflkz" event={"ID":"c0e3ecc0-e47b-4612-87aa-53ce8434f5a8","Type":"ContainerStarted","Data":"2adf155696b232b2899ec00c5819b70471f16cbd2a93242b54c8af78d3d70440"}
Nov 02 11:56:33.686870 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:33.686843    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm" event={"ID":"f38c9faa-5fae-45ba-93f6-e252b6f092fd","Type":"ContainerStarted","Data":"f82ca0b279a815dfc1b811fb8f503c3c7377927492e07d47e849176f5935137a"}
Nov 02 11:56:33.687004 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:33.686890    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-9vksm" event={"ID":"f38c9faa-5fae-45ba-93f6-e252b6f092fd","Type":"ContainerStarted","Data":"7a3e1862c82f921c8805cdedbc5f4c251eddee783615deac58bce509f967534f"}
Nov 02 11:56:34.692501 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:34.692460    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-krn29" event={"ID":"252df0cc-a5c0-4dea-aca5-bd6b08262d38","Type":"ContainerStarted","Data":"c60ef4890ef9e01ed3ba0c86029b9f9ca1d120019dc50080d23e2d8c4c55d3ed"}
Nov 02 11:56:35.648595 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:35.648552    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cttl7"
Nov 02 11:56:42.034877 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:42.034842    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:43.418145 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:43.418117    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:56:44.774800 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:44.774775    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7686t"
Nov 02 11:56:44.817178 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:44.817152    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7pvw8"]
Nov 02 11:56:44.841285 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:44.841256    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:44.924035 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:44.923998    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32151a32-010f-470e-bf7f-80122635fa2e-node-exporter-textfile\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:44.924182 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:44.924088    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32151a32-010f-470e-bf7f-80122635fa2e-root\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:44.924182 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:44.924125    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32151a32-010f-470e-bf7f-80122635fa2e-node-exporter-accelerators-collector-config\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:44.924182 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:44.924150    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32151a32-010f-470e-bf7f-80122635fa2e-node-exporter-tls\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:44.924182 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:44.924165    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32151a32-010f-470e-bf7f-80122635fa2e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:44.924399 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:44.924190    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32151a32-010f-470e-bf7f-80122635fa2e-sys\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:44.924399 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:44.924259    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bqvm\" (UniqueName: \"kubernetes.io/projected/32151a32-010f-470e-bf7f-80122635fa2e-kube-api-access-8bqvm\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:44.924501 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:44.924449    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32151a32-010f-470e-bf7f-80122635fa2e-metrics-client-ca\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:44.924501 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:44.924485    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32151a32-010f-470e-bf7f-80122635fa2e-node-exporter-wtmp\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:44.968490 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:44.968465    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 02 11:56:44.968490 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:44.968480    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 02 11:56:44.968490 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:44.968492    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-brq6w\""
Nov 02 11:56:44.968996 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:44.968980    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 02 11:56:44.969038 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:44.969010    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 02 11:56:45.025143 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.025088    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32151a32-010f-470e-bf7f-80122635fa2e-metrics-client-ca\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.025143 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.025116    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32151a32-010f-470e-bf7f-80122635fa2e-node-exporter-wtmp\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.025143 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.025142    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32151a32-010f-470e-bf7f-80122635fa2e-node-exporter-textfile\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.025343 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.025253    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/32151a32-010f-470e-bf7f-80122635fa2e-node-exporter-wtmp\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.025343 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.025254    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32151a32-010f-470e-bf7f-80122635fa2e-root\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.025343 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.025286    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/32151a32-010f-470e-bf7f-80122635fa2e-root\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.025343 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.025321    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32151a32-010f-470e-bf7f-80122635fa2e-node-exporter-accelerators-collector-config\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.025501 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.025359    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32151a32-010f-470e-bf7f-80122635fa2e-node-exporter-tls\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.025501 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.025379    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32151a32-010f-470e-bf7f-80122635fa2e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.025501 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.025405    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32151a32-010f-470e-bf7f-80122635fa2e-sys\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.025501 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.025426    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bqvm\" (UniqueName: \"kubernetes.io/projected/32151a32-010f-470e-bf7f-80122635fa2e-kube-api-access-8bqvm\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.025684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.025518    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/32151a32-010f-470e-bf7f-80122635fa2e-sys\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.025684 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.025647    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/32151a32-010f-470e-bf7f-80122635fa2e-node-exporter-textfile\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.025809 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.025787    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32151a32-010f-470e-bf7f-80122635fa2e-metrics-client-ca\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.025866 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.025807    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/32151a32-010f-470e-bf7f-80122635fa2e-node-exporter-accelerators-collector-config\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.027746 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.027726    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/32151a32-010f-470e-bf7f-80122635fa2e-node-exporter-tls\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.027842 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.027760    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32151a32-010f-470e-bf7f-80122635fa2e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.088571 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.088542    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bqvm\" (UniqueName: \"kubernetes.io/projected/32151a32-010f-470e-bf7f-80122635fa2e-kube-api-access-8bqvm\") pod \"node-exporter-7pvw8\" (UID: \"32151a32-010f-470e-bf7f-80122635fa2e\") " pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.182428 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.182396    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7pvw8"
Nov 02 11:56:45.215854 ip-10-0-135-24 kubenswrapper[4942]: W1102 11:56:45.215826    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32151a32_010f_470e_bf7f_80122635fa2e.slice/crio-473c629c89f68883f4352f38dbfa798c1c27ac56d7aeea8e299d73d7965b7508 WatchSource:0}: Error finding container 473c629c89f68883f4352f38dbfa798c1c27ac56d7aeea8e299d73d7965b7508: Status 404 returned error can't find the container with id 473c629c89f68883f4352f38dbfa798c1c27ac56d7aeea8e299d73d7965b7508
Nov 02 11:56:45.730289 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:45.730253    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7pvw8" event={"ID":"32151a32-010f-470e-bf7f-80122635fa2e","Type":"ContainerStarted","Data":"473c629c89f68883f4352f38dbfa798c1c27ac56d7aeea8e299d73d7965b7508"}
Nov 02 11:56:46.906791 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:46.906761    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2vw9j"
Nov 02 11:56:47.738952 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:47.738922    4942 generic.go:356] "Generic (PLEG): container finished" podID="32151a32-010f-470e-bf7f-80122635fa2e" containerID="bd024d9c96c0b4af61e719ef8f74712ced7cea00311ef6b64e40e0e8d844e400" exitCode=0
Nov 02 11:56:47.739194 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:47.739004    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7pvw8" event={"ID":"32151a32-010f-470e-bf7f-80122635fa2e","Type":"ContainerDied","Data":"bd024d9c96c0b4af61e719ef8f74712ced7cea00311ef6b64e40e0e8d844e400"}
Nov 02 11:56:48.745381 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:48.745342    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7pvw8" event={"ID":"32151a32-010f-470e-bf7f-80122635fa2e","Type":"ContainerStarted","Data":"8912dbdb6d155c4cfddef8bc817d98dfde36c9f31b2a124c1e2ee224fdbc7e68"}
Nov 02 11:56:48.745862 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:48.745404    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7pvw8" event={"ID":"32151a32-010f-470e-bf7f-80122635fa2e","Type":"ContainerStarted","Data":"22dccb1a42512d1d606e087e07bf200173ea607565505f3c1d679ba4ef6ff3d5"}
Nov 02 11:56:48.913703 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:48.913651    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7pvw8" podStartSLOduration=3.103195181 podStartE2EDuration="4.913636406s" totalImagesPullingTime="1.735710013s" totalInitContainerRuntime="74.731212ms" isStatefulPod=true podCreationTimestamp="2026-11-02 11:56:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 11:56:48.91349076 +0000 UTC m=+35.484840512" watchObservedRunningTime="2026-11-02 11:56:48.913636406 +0000 UTC m=+35.484986157"
Nov 02 11:56:50.760374 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:50.760338    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d58f4b759-ts6c2"]
Nov 02 11:56:57.060373 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.060260    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx" podUID="2759fb5a-17ad-4c29-b9c8-175769f53069" containerName="registry" containerID="cri-o://51cd52accb53dd63a7aa5d4e18d45d0e007076c89f32a1db782ed015b71ce0ff" gracePeriod=30
Nov 02 11:56:57.332011 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.331955    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:57.428092 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.428068    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-installation-pull-secrets\") pod \"2759fb5a-17ad-4c29-b9c8-175769f53069\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") "
Nov 02 11:56:57.428208 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.428101    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-trusted-ca\") pod \"2759fb5a-17ad-4c29-b9c8-175769f53069\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") "
Nov 02 11:56:57.428208 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.428156    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-bound-sa-token\") pod \"2759fb5a-17ad-4c29-b9c8-175769f53069\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") "
Nov 02 11:56:57.428208 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.428181    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2759fb5a-17ad-4c29-b9c8-175769f53069-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2759fb5a-17ad-4c29-b9c8-175769f53069-ca-trust-extracted\") pod \"2759fb5a-17ad-4c29-b9c8-175769f53069\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") "
Nov 02 11:56:57.428389 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.428215    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls\") pod \"2759fb5a-17ad-4c29-b9c8-175769f53069\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") "
Nov 02 11:56:57.428389 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.428368    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-certificates\") pod \"2759fb5a-17ad-4c29-b9c8-175769f53069\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") "
Nov 02 11:56:57.428494 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.428435    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-image-registry-private-configuration\") pod \"2759fb5a-17ad-4c29-b9c8-175769f53069\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") "
Nov 02 11:56:57.428550 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.428505    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-kube-api-access-5mg2c\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-kube-api-access-5mg2c\") pod \"2759fb5a-17ad-4c29-b9c8-175769f53069\" (UID: \"2759fb5a-17ad-4c29-b9c8-175769f53069\") "
Nov 02 11:56:57.428955 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.428720    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-trusted-ca" pod "2759fb5a-17ad-4c29-b9c8-175769f53069" (UID: "2759fb5a-17ad-4c29-b9c8-175769f53069"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 11:56:57.428955 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.428882    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-certificates" pod "2759fb5a-17ad-4c29-b9c8-175769f53069" (UID: "2759fb5a-17ad-4c29-b9c8-175769f53069"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 11:56:57.430836 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.430790    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-installation-pull-secrets" pod "2759fb5a-17ad-4c29-b9c8-175769f53069" (UID: "2759fb5a-17ad-4c29-b9c8-175769f53069"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 11:56:57.431109 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.431081    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls" pod "2759fb5a-17ad-4c29-b9c8-175769f53069" (UID: "2759fb5a-17ad-4c29-b9c8-175769f53069"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 11:56:57.431213 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.431196    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-bound-sa-token" pod "2759fb5a-17ad-4c29-b9c8-175769f53069" (UID: "2759fb5a-17ad-4c29-b9c8-175769f53069"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 11:56:57.431277 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.431231    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-image-registry-private-configuration" pod "2759fb5a-17ad-4c29-b9c8-175769f53069" (UID: "2759fb5a-17ad-4c29-b9c8-175769f53069"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 11:56:57.431407 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.431369    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-kube-api-access-5mg2c" pod "2759fb5a-17ad-4c29-b9c8-175769f53069" (UID: "2759fb5a-17ad-4c29-b9c8-175769f53069"). InnerVolumeSpecName "kube-api-access-5mg2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 11:56:57.435330 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.435291    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2759fb5a-17ad-4c29-b9c8-175769f53069-ca-trust-extracted" pod "2759fb5a-17ad-4c29-b9c8-175769f53069" (UID: "2759fb5a-17ad-4c29-b9c8-175769f53069"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 11:56:57.530106 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.530070    4942 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-bound-sa-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 11:56:57.530249 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.530110    4942 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2759fb5a-17ad-4c29-b9c8-175769f53069-ca-trust-extracted\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 11:56:57.530249 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.530126    4942 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 11:56:57.530249 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.530141    4942 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-registry-certificates\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 11:56:57.530249 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.530159    4942 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-image-registry-private-configuration\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 11:56:57.530249 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.530174    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mg2c\" (UniqueName: \"kubernetes.io/projected/2759fb5a-17ad-4c29-b9c8-175769f53069-kube-api-access-5mg2c\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 11:56:57.530249 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.530189    4942 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2759fb5a-17ad-4c29-b9c8-175769f53069-installation-pull-secrets\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 11:56:57.530249 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.530204    4942 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2759fb5a-17ad-4c29-b9c8-175769f53069-trusted-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 11:56:57.778894 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.778855    4942 generic.go:356] "Generic (PLEG): container finished" podID="2759fb5a-17ad-4c29-b9c8-175769f53069" containerID="51cd52accb53dd63a7aa5d4e18d45d0e007076c89f32a1db782ed015b71ce0ff" exitCode=0
Nov 02 11:56:57.779085 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.778907    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx"
Nov 02 11:56:57.779085 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.778938    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx" event={"ID":"2759fb5a-17ad-4c29-b9c8-175769f53069","Type":"ContainerDied","Data":"51cd52accb53dd63a7aa5d4e18d45d0e007076c89f32a1db782ed015b71ce0ff"}
Nov 02 11:56:57.779085 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.778993    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75766bbfd6-gpgxx" event={"ID":"2759fb5a-17ad-4c29-b9c8-175769f53069","Type":"ContainerDied","Data":"1691d6cc3707d6adf663623b810246ea6f41ccde3d2b1a8a477e173e507aaf4c"}
Nov 02 11:56:57.779085 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.779014    4942 scope.go:122] "RemoveContainer" containerID="51cd52accb53dd63a7aa5d4e18d45d0e007076c89f32a1db782ed015b71ce0ff"
Nov 02 11:56:57.786777 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.786758    4942 scope.go:122] "RemoveContainer" containerID="51cd52accb53dd63a7aa5d4e18d45d0e007076c89f32a1db782ed015b71ce0ff"
Nov 02 11:56:57.787034 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:56:57.787014    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51cd52accb53dd63a7aa5d4e18d45d0e007076c89f32a1db782ed015b71ce0ff\": container with ID starting with 51cd52accb53dd63a7aa5d4e18d45d0e007076c89f32a1db782ed015b71ce0ff not found: ID does not exist" containerID="51cd52accb53dd63a7aa5d4e18d45d0e007076c89f32a1db782ed015b71ce0ff"
Nov 02 11:56:57.787080 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.787042    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51cd52accb53dd63a7aa5d4e18d45d0e007076c89f32a1db782ed015b71ce0ff"} err="failed to get container status \"51cd52accb53dd63a7aa5d4e18d45d0e007076c89f32a1db782ed015b71ce0ff\": rpc error: code = NotFound desc = could not find container \"51cd52accb53dd63a7aa5d4e18d45d0e007076c89f32a1db782ed015b71ce0ff\": container with ID starting with 51cd52accb53dd63a7aa5d4e18d45d0e007076c89f32a1db782ed015b71ce0ff not found: ID does not exist"
Nov 02 11:56:57.941787 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.941753    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75766bbfd6-gpgxx"]
Nov 02 11:56:57.990657 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:57.990629    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-75766bbfd6-gpgxx"]
Nov 02 11:56:59.540390 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:56:59.540359    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2759fb5a-17ad-4c29-b9c8-175769f53069" path="/var/lib/kubelet/pods/2759fb5a-17ad-4c29-b9c8-175769f53069/volumes"
Nov 02 11:57:00.983021 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:57:00.982986    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 11:57:15.779406 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:15.779344    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2" podUID="e9dc2cae-7e99-4282-a02d-51fb0f12b43b" containerName="registry" containerID="cri-o://c21c939fbd73af2997869fe4e2d17bb78541c62b87ca50a0af6bc652d6d0cee2" gracePeriod=30
Nov 02 11:57:16.027845 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.027822    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:57:16.174065 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.174038    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-ca-trust-extracted\") pod \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") "
Nov 02 11:57:16.174250 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.174097    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-kube-api-access-vqzcg\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-kube-api-access-vqzcg\") pod \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") "
Nov 02 11:57:16.174250 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.174126    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls\") pod \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") "
Nov 02 11:57:16.174250 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.174156    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-installation-pull-secrets\") pod \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") "
Nov 02 11:57:16.174250 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.174185    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-bound-sa-token\") pod \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") "
Nov 02 11:57:16.174250 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.174212    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-image-registry-private-configuration\") pod \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") "
Nov 02 11:57:16.174556 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.174257    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-certificates\") pod \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") "
Nov 02 11:57:16.174556 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.174317    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-trusted-ca\") pod \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\" (UID: \"e9dc2cae-7e99-4282-a02d-51fb0f12b43b\") "
Nov 02 11:57:16.174920 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.174895    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-trusted-ca" pod "e9dc2cae-7e99-4282-a02d-51fb0f12b43b" (UID: "e9dc2cae-7e99-4282-a02d-51fb0f12b43b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 11:57:16.175022 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.174912    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-certificates" pod "e9dc2cae-7e99-4282-a02d-51fb0f12b43b" (UID: "e9dc2cae-7e99-4282-a02d-51fb0f12b43b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 11:57:16.176954 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.176922    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-image-registry-private-configuration" pod "e9dc2cae-7e99-4282-a02d-51fb0f12b43b" (UID: "e9dc2cae-7e99-4282-a02d-51fb0f12b43b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 11:57:16.176954 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.176933    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-bound-sa-token" pod "e9dc2cae-7e99-4282-a02d-51fb0f12b43b" (UID: "e9dc2cae-7e99-4282-a02d-51fb0f12b43b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 11:57:16.177125 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.177044    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-installation-pull-secrets" pod "e9dc2cae-7e99-4282-a02d-51fb0f12b43b" (UID: "e9dc2cae-7e99-4282-a02d-51fb0f12b43b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 11:57:16.177125 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.177053    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-kube-api-access-vqzcg" pod "e9dc2cae-7e99-4282-a02d-51fb0f12b43b" (UID: "e9dc2cae-7e99-4282-a02d-51fb0f12b43b"). InnerVolumeSpecName "kube-api-access-vqzcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 11:57:16.177193 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.177157    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls" pod "e9dc2cae-7e99-4282-a02d-51fb0f12b43b" (UID: "e9dc2cae-7e99-4282-a02d-51fb0f12b43b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 11:57:16.181575 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.181549    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-ca-trust-extracted" pod "e9dc2cae-7e99-4282-a02d-51fb0f12b43b" (UID: "e9dc2cae-7e99-4282-a02d-51fb0f12b43b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 11:57:16.275851 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.275829    4942 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-certificates\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 11:57:16.275851 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.275851    4942 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-trusted-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 11:57:16.275993 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.275861    4942 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-ca-trust-extracted\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 11:57:16.275993 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.275871    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqzcg\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-kube-api-access-vqzcg\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 11:57:16.275993 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.275879    4942 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-registry-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 11:57:16.275993 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.275888    4942 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-installation-pull-secrets\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 11:57:16.275993 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.275897    4942 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-bound-sa-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 11:57:16.275993 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.275907    4942 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e9dc2cae-7e99-4282-a02d-51fb0f12b43b-image-registry-private-configuration\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 11:57:16.852537 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.852500    4942 generic.go:356] "Generic (PLEG): container finished" podID="e9dc2cae-7e99-4282-a02d-51fb0f12b43b" containerID="c21c939fbd73af2997869fe4e2d17bb78541c62b87ca50a0af6bc652d6d0cee2" exitCode=0
Nov 02 11:57:16.852930 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.852553    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2"
Nov 02 11:57:16.852930 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.852566    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2" event={"ID":"e9dc2cae-7e99-4282-a02d-51fb0f12b43b","Type":"ContainerDied","Data":"c21c939fbd73af2997869fe4e2d17bb78541c62b87ca50a0af6bc652d6d0cee2"}
Nov 02 11:57:16.852930 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.852617    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d58f4b759-ts6c2" event={"ID":"e9dc2cae-7e99-4282-a02d-51fb0f12b43b","Type":"ContainerDied","Data":"3451f27745f1ab8a2b397a8f00ac679a44a3948c2bd9206528ec990edf6eee33"}
Nov 02 11:57:16.852930 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.852645    4942 scope.go:122] "RemoveContainer" containerID="c21c939fbd73af2997869fe4e2d17bb78541c62b87ca50a0af6bc652d6d0cee2"
Nov 02 11:57:16.861358 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.861336    4942 scope.go:122] "RemoveContainer" containerID="c21c939fbd73af2997869fe4e2d17bb78541c62b87ca50a0af6bc652d6d0cee2"
Nov 02 11:57:16.861619 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:57:16.861598    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c21c939fbd73af2997869fe4e2d17bb78541c62b87ca50a0af6bc652d6d0cee2\": container with ID starting with c21c939fbd73af2997869fe4e2d17bb78541c62b87ca50a0af6bc652d6d0cee2 not found: ID does not exist" containerID="c21c939fbd73af2997869fe4e2d17bb78541c62b87ca50a0af6bc652d6d0cee2"
Nov 02 11:57:16.861683 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:16.861631    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c21c939fbd73af2997869fe4e2d17bb78541c62b87ca50a0af6bc652d6d0cee2"} err="failed to get container status \"c21c939fbd73af2997869fe4e2d17bb78541c62b87ca50a0af6bc652d6d0cee2\": rpc error: code = NotFound desc = could not find container \"c21c939fbd73af2997869fe4e2d17bb78541c62b87ca50a0af6bc652d6d0cee2\": container with ID starting with c21c939fbd73af2997869fe4e2d17bb78541c62b87ca50a0af6bc652d6d0cee2 not found: ID does not exist"
Nov 02 11:57:17.065579 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:17.065550    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d58f4b759-ts6c2"]
Nov 02 11:57:17.115497 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:17.115431    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6d58f4b759-ts6c2"]
Nov 02 11:57:17.541050 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:17.540977    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9dc2cae-7e99-4282-a02d-51fb0f12b43b" path="/var/lib/kubelet/pods/e9dc2cae-7e99-4282-a02d-51fb0f12b43b/volumes"
Nov 02 11:57:19.866440 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:19.866408    4942 generic.go:356] "Generic (PLEG): container finished" podID="6384f50d-3bbe-47ef-8e56-5a72d8113c6f" containerID="9780fb1d56c0397763b010f610fc1aa2b0a413e0df184563c15d73117ac9ed9d" exitCode=0
Nov 02 11:57:19.866795 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:19.866472    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4" event={"ID":"6384f50d-3bbe-47ef-8e56-5a72d8113c6f","Type":"ContainerDied","Data":"9780fb1d56c0397763b010f610fc1aa2b0a413e0df184563c15d73117ac9ed9d"}
Nov 02 11:57:19.866795 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:19.866752    4942 scope.go:122] "RemoveContainer" containerID="9780fb1d56c0397763b010f610fc1aa2b0a413e0df184563c15d73117ac9ed9d"
Nov 02 11:57:20.872712 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:20.872678    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-9mfg4" event={"ID":"6384f50d-3bbe-47ef-8e56-5a72d8113c6f","Type":"ContainerStarted","Data":"7cd1926ac01e07385d6889fa9274819c46de3f5a9039991c4161598410762f1b"}
Nov 02 11:57:30.982042 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:57:30.981999    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 11:57:34.920262 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:34.920229    4942 generic.go:356] "Generic (PLEG): container finished" podID="691a1fbe-dd29-4579-8e72-8c36c2b1eda2" containerID="7978ff5a32a1fb97eaba5670abb89fa6708f6695a68dadc060145d954d13a237" exitCode=0
Nov 02 11:57:34.920667 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:34.920277    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf" event={"ID":"691a1fbe-dd29-4579-8e72-8c36c2b1eda2","Type":"ContainerDied","Data":"7978ff5a32a1fb97eaba5670abb89fa6708f6695a68dadc060145d954d13a237"}
Nov 02 11:57:34.920667 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:34.920584    4942 scope.go:122] "RemoveContainer" containerID="7978ff5a32a1fb97eaba5670abb89fa6708f6695a68dadc060145d954d13a237"
Nov 02 11:57:35.925248 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:35.925215    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-fgmpf" event={"ID":"691a1fbe-dd29-4579-8e72-8c36c2b1eda2","Type":"ContainerStarted","Data":"da98f9c5da8e5887f61e7b5508387671950ae60ea1e94771c609a19a3d7e5bee"}
Nov 02 11:57:39.940364 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:39.940326    4942 generic.go:356] "Generic (PLEG): container finished" podID="9176bf1e-471e-43a2-af6b-650c5a494efa" containerID="145d3b348fa0332337a96851d2937b4a9b2550767b90a22c53c352510130596b" exitCode=0
Nov 02 11:57:39.940739 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:39.940395    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-rhl9j" event={"ID":"9176bf1e-471e-43a2-af6b-650c5a494efa","Type":"ContainerDied","Data":"145d3b348fa0332337a96851d2937b4a9b2550767b90a22c53c352510130596b"}
Nov 02 11:57:39.940739 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:39.940701    4942 scope.go:122] "RemoveContainer" containerID="145d3b348fa0332337a96851d2937b4a9b2550767b90a22c53c352510130596b"
Nov 02 11:57:40.945945 ip-10-0-135-24 kubenswrapper[4942]: I1102 11:57:40.945910    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-rhl9j" event={"ID":"9176bf1e-471e-43a2-af6b-650c5a494efa","Type":"ContainerStarted","Data":"9894725140e5a4d033bb785f1f77dec3a10d8155178f859162f2630c4bcf95a6"}
Nov 02 11:58:00.982074 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:58:00.982037    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 11:58:30.981983 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:58:30.981910    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 11:59:00.981458 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:59:00.981421    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 11:59:30.980837 ip-10-0-135-24 kubenswrapper[4942]: E1102 11:59:30.980803    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:00:00.982442 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:00:00.982348    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:00:30.983463 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:00:30.983426    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:01:00.980745 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:01:00.980707    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:01:13.565821 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:13.565789    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:01:13.566322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:13.566242    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:01:20.174990 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.174956    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5"]
Nov 02 12:01:20.175386 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.175239    4942 cpu_manager.go:414] "containerMap: removing container" podUID="e9dc2cae-7e99-4282-a02d-51fb0f12b43b" containerName="registry"
Nov 02 12:01:20.175386 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.175250    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9dc2cae-7e99-4282-a02d-51fb0f12b43b" containerName="registry"
Nov 02 12:01:20.175386 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.175261    4942 cpu_manager.go:414] "containerMap: removing container" podUID="2759fb5a-17ad-4c29-b9c8-175769f53069" containerName="registry"
Nov 02 12:01:20.175386 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.175268    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2759fb5a-17ad-4c29-b9c8-175769f53069" containerName="registry"
Nov 02 12:01:20.175386 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.175336    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9dc2cae-7e99-4282-a02d-51fb0f12b43b" containerName="registry"
Nov 02 12:01:20.175386 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.175344    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="2759fb5a-17ad-4c29-b9c8-175769f53069" containerName="registry"
Nov 02 12:01:20.178441 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.178419    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5"
Nov 02 12:01:20.279802 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.279771    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 02 12:01:20.279802 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.279771    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 02 12:01:20.280000 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.279868    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-2hncp\""
Nov 02 12:01:20.280881 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.280864    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5"]
Nov 02 12:01:20.343997 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.343972    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a322fc87-6c9f-40a4-9d90-90a370bcdb01-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5\" (UID: \"a322fc87-6c9f-40a4-9d90-90a370bcdb01\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5"
Nov 02 12:01:20.344148 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.344028    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt9ck\" (UniqueName: \"kubernetes.io/projected/a322fc87-6c9f-40a4-9d90-90a370bcdb01-kube-api-access-mt9ck\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5\" (UID: \"a322fc87-6c9f-40a4-9d90-90a370bcdb01\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5"
Nov 02 12:01:20.344148 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.344131    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a322fc87-6c9f-40a4-9d90-90a370bcdb01-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5\" (UID: \"a322fc87-6c9f-40a4-9d90-90a370bcdb01\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5"
Nov 02 12:01:20.444636 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.444581    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a322fc87-6c9f-40a4-9d90-90a370bcdb01-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5\" (UID: \"a322fc87-6c9f-40a4-9d90-90a370bcdb01\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5"
Nov 02 12:01:20.444636 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.444635    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt9ck\" (UniqueName: \"kubernetes.io/projected/a322fc87-6c9f-40a4-9d90-90a370bcdb01-kube-api-access-mt9ck\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5\" (UID: \"a322fc87-6c9f-40a4-9d90-90a370bcdb01\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5"
Nov 02 12:01:20.444812 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.444663    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a322fc87-6c9f-40a4-9d90-90a370bcdb01-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5\" (UID: \"a322fc87-6c9f-40a4-9d90-90a370bcdb01\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5"
Nov 02 12:01:20.444961 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.444943    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a322fc87-6c9f-40a4-9d90-90a370bcdb01-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5\" (UID: \"a322fc87-6c9f-40a4-9d90-90a370bcdb01\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5"
Nov 02 12:01:20.445036 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.445003    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a322fc87-6c9f-40a4-9d90-90a370bcdb01-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5\" (UID: \"a322fc87-6c9f-40a4-9d90-90a370bcdb01\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5"
Nov 02 12:01:20.505239 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.505211    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt9ck\" (UniqueName: \"kubernetes.io/projected/a322fc87-6c9f-40a4-9d90-90a370bcdb01-kube-api-access-mt9ck\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5\" (UID: \"a322fc87-6c9f-40a4-9d90-90a370bcdb01\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5"
Nov 02 12:01:20.604602 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.604574    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk"]
Nov 02 12:01:20.607945 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.607926    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk"
Nov 02 12:01:20.659968 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.659950    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-krtmk\""
Nov 02 12:01:20.666553 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.666523    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk"]
Nov 02 12:01:20.702876 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.702826    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm"]
Nov 02 12:01:20.706061 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.706045    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm"
Nov 02 12:01:20.747661 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.747634    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cca579e9-688a-4e73-8682-1a30b418ad54-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk\" (UID: \"cca579e9-688a-4e73-8682-1a30b418ad54\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk"
Nov 02 12:01:20.747758 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.747671    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ks7c\" (UniqueName: \"kubernetes.io/projected/cca579e9-688a-4e73-8682-1a30b418ad54-kube-api-access-4ks7c\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk\" (UID: \"cca579e9-688a-4e73-8682-1a30b418ad54\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk"
Nov 02 12:01:20.747758 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.747740    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cca579e9-688a-4e73-8682-1a30b418ad54-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk\" (UID: \"cca579e9-688a-4e73-8682-1a30b418ad54\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk"
Nov 02 12:01:20.760154 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.760134    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-jmnvh\""
Nov 02 12:01:20.779654 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.779630    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm"]
Nov 02 12:01:20.793342 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.793323    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5"
Nov 02 12:01:20.796333 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.796184    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw"]
Nov 02 12:01:20.799481 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.799466    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw"
Nov 02 12:01:20.848500 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.848474    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cca579e9-688a-4e73-8682-1a30b418ad54-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk\" (UID: \"cca579e9-688a-4e73-8682-1a30b418ad54\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk"
Nov 02 12:01:20.848673 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.848651    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b556fe62-b13c-4c41-b452-9543d5dc01d7-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm\" (UID: \"b556fe62-b13c-4c41-b452-9543d5dc01d7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm"
Nov 02 12:01:20.848830 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.848811    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2llm\" (UniqueName: \"kubernetes.io/projected/b556fe62-b13c-4c41-b452-9543d5dc01d7-kube-api-access-r2llm\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm\" (UID: \"b556fe62-b13c-4c41-b452-9543d5dc01d7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm"
Nov 02 12:01:20.848941 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.848921    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cca579e9-688a-4e73-8682-1a30b418ad54-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk\" (UID: \"cca579e9-688a-4e73-8682-1a30b418ad54\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk"
Nov 02 12:01:20.849045 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.849027    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b556fe62-b13c-4c41-b452-9543d5dc01d7-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm\" (UID: \"b556fe62-b13c-4c41-b452-9543d5dc01d7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm"
Nov 02 12:01:20.849200 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.849182    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cca579e9-688a-4e73-8682-1a30b418ad54-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk\" (UID: \"cca579e9-688a-4e73-8682-1a30b418ad54\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk"
Nov 02 12:01:20.849398 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.849377    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ks7c\" (UniqueName: \"kubernetes.io/projected/cca579e9-688a-4e73-8682-1a30b418ad54-kube-api-access-4ks7c\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk\" (UID: \"cca579e9-688a-4e73-8682-1a30b418ad54\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk"
Nov 02 12:01:20.849539 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.849463    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cca579e9-688a-4e73-8682-1a30b418ad54-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk\" (UID: \"cca579e9-688a-4e73-8682-1a30b418ad54\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk"
Nov 02 12:01:20.858273 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.858225    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-7krdx\""
Nov 02 12:01:20.858614 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.858591    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw"]
Nov 02 12:01:20.903958 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.903930    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c"]
Nov 02 12:01:20.907517 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.907498    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c"
Nov 02 12:01:20.913139 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.913120    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ks7c\" (UniqueName: \"kubernetes.io/projected/cca579e9-688a-4e73-8682-1a30b418ad54-kube-api-access-4ks7c\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk\" (UID: \"cca579e9-688a-4e73-8682-1a30b418ad54\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk"
Nov 02 12:01:20.923113 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.923095    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk"
Nov 02 12:01:20.950483 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.950452    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2llm\" (UniqueName: \"kubernetes.io/projected/b556fe62-b13c-4c41-b452-9543d5dc01d7-kube-api-access-r2llm\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm\" (UID: \"b556fe62-b13c-4c41-b452-9543d5dc01d7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm"
Nov 02 12:01:20.950596 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.950519    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b556fe62-b13c-4c41-b452-9543d5dc01d7-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm\" (UID: \"b556fe62-b13c-4c41-b452-9543d5dc01d7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm"
Nov 02 12:01:20.950655 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.950639    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f95cad35-61f5-4419-9533-c2f9add0d73e-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw\" (UID: \"f95cad35-61f5-4419-9533-c2f9add0d73e\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw"
Nov 02 12:01:20.950700 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.950688    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2stdk\" (UniqueName: \"kubernetes.io/projected/f95cad35-61f5-4419-9533-c2f9add0d73e-kube-api-access-2stdk\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw\" (UID: \"f95cad35-61f5-4419-9533-c2f9add0d73e\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw"
Nov 02 12:01:20.951048 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.950847    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f95cad35-61f5-4419-9533-c2f9add0d73e-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw\" (UID: \"f95cad35-61f5-4419-9533-c2f9add0d73e\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw"
Nov 02 12:01:20.951048 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.950890    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b556fe62-b13c-4c41-b452-9543d5dc01d7-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm\" (UID: \"b556fe62-b13c-4c41-b452-9543d5dc01d7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm"
Nov 02 12:01:20.951048 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.950897    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b556fe62-b13c-4c41-b452-9543d5dc01d7-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm\" (UID: \"b556fe62-b13c-4c41-b452-9543d5dc01d7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm"
Nov 02 12:01:20.951248 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.951155    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b556fe62-b13c-4c41-b452-9543d5dc01d7-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm\" (UID: \"b556fe62-b13c-4c41-b452-9543d5dc01d7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm"
Nov 02 12:01:20.959579 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.959511    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-rjvtq\""
Nov 02 12:01:20.965612 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:20.965291    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c"]
Nov 02 12:01:21.011205 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.011181    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2llm\" (UniqueName: \"kubernetes.io/projected/b556fe62-b13c-4c41-b452-9543d5dc01d7-kube-api-access-r2llm\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm\" (UID: \"b556fe62-b13c-4c41-b452-9543d5dc01d7\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm"
Nov 02 12:01:21.020987 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.020968    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm"
Nov 02 12:01:21.052008 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.051974    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f95cad35-61f5-4419-9533-c2f9add0d73e-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw\" (UID: \"f95cad35-61f5-4419-9533-c2f9add0d73e\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw"
Nov 02 12:01:21.052131 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.052027    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53c00485-1170-4977-919a-8d463b407845-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c\" (UID: \"53c00485-1170-4977-919a-8d463b407845\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c"
Nov 02 12:01:21.052131 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.052077    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6w4x\" (UniqueName: \"kubernetes.io/projected/53c00485-1170-4977-919a-8d463b407845-kube-api-access-h6w4x\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c\" (UID: \"53c00485-1170-4977-919a-8d463b407845\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c"
Nov 02 12:01:21.052131 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.052124    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53c00485-1170-4977-919a-8d463b407845-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c\" (UID: \"53c00485-1170-4977-919a-8d463b407845\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c"
Nov 02 12:01:21.052227 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.052165    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f95cad35-61f5-4419-9533-c2f9add0d73e-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw\" (UID: \"f95cad35-61f5-4419-9533-c2f9add0d73e\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw"
Nov 02 12:01:21.052227 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.052205    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2stdk\" (UniqueName: \"kubernetes.io/projected/f95cad35-61f5-4419-9533-c2f9add0d73e-kube-api-access-2stdk\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw\" (UID: \"f95cad35-61f5-4419-9533-c2f9add0d73e\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw"
Nov 02 12:01:21.052407 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.052386    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f95cad35-61f5-4419-9533-c2f9add0d73e-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw\" (UID: \"f95cad35-61f5-4419-9533-c2f9add0d73e\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw"
Nov 02 12:01:21.052656 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.052635    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f95cad35-61f5-4419-9533-c2f9add0d73e-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw\" (UID: \"f95cad35-61f5-4419-9533-c2f9add0d73e\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw"
Nov 02 12:01:21.115383 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.115354    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2stdk\" (UniqueName: \"kubernetes.io/projected/f95cad35-61f5-4419-9533-c2f9add0d73e-kube-api-access-2stdk\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw\" (UID: \"f95cad35-61f5-4419-9533-c2f9add0d73e\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw"
Nov 02 12:01:21.129144 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.129121    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw"
Nov 02 12:01:21.153358 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.153336    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53c00485-1170-4977-919a-8d463b407845-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c\" (UID: \"53c00485-1170-4977-919a-8d463b407845\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c"
Nov 02 12:01:21.153478 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.153375    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6w4x\" (UniqueName: \"kubernetes.io/projected/53c00485-1170-4977-919a-8d463b407845-kube-api-access-h6w4x\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c\" (UID: \"53c00485-1170-4977-919a-8d463b407845\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c"
Nov 02 12:01:21.153544 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.153495    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53c00485-1170-4977-919a-8d463b407845-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c\" (UID: \"53c00485-1170-4977-919a-8d463b407845\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c"
Nov 02 12:01:21.153773 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.153750    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53c00485-1170-4977-919a-8d463b407845-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c\" (UID: \"53c00485-1170-4977-919a-8d463b407845\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c"
Nov 02 12:01:21.153840 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.153776    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53c00485-1170-4977-919a-8d463b407845-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c\" (UID: \"53c00485-1170-4977-919a-8d463b407845\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c"
Nov 02 12:01:21.214484 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.214431    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6w4x\" (UniqueName: \"kubernetes.io/projected/53c00485-1170-4977-919a-8d463b407845-kube-api-access-h6w4x\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c\" (UID: \"53c00485-1170-4977-919a-8d463b407845\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c"
Nov 02 12:01:21.218998 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.218976    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5"]
Nov 02 12:01:21.221338 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:01:21.221314    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda322fc87_6c9f_40a4_9d90_90a370bcdb01.slice/crio-bfa5069ff79dfe799546dfc5318b42a59483a751c128f3155e01290f35f37253 WatchSource:0}: Error finding container bfa5069ff79dfe799546dfc5318b42a59483a751c128f3155e01290f35f37253: Status 404 returned error can't find the container with id bfa5069ff79dfe799546dfc5318b42a59483a751c128f3155e01290f35f37253
Nov 02 12:01:21.229989 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.229968    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c"
Nov 02 12:01:21.296868 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.296846    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk"]
Nov 02 12:01:21.298778 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:01:21.298751    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcca579e9_688a_4e73_8682_1a30b418ad54.slice/crio-8fdcd81b8e79bc0b692cde70841cece1d896cda91145fba767b2e6036d4c2bd2 WatchSource:0}: Error finding container 8fdcd81b8e79bc0b692cde70841cece1d896cda91145fba767b2e6036d4c2bd2: Status 404 returned error can't find the container with id 8fdcd81b8e79bc0b692cde70841cece1d896cda91145fba767b2e6036d4c2bd2
Nov 02 12:01:21.390805 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.390776    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm"]
Nov 02 12:01:21.393202 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:01:21.393179    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb556fe62_b13c_4c41_b452_9543d5dc01d7.slice/crio-dcfb7b3dc43a797336d557912f7b51bf09f6b8953fc80285d60040cc98ddad14 WatchSource:0}: Error finding container dcfb7b3dc43a797336d557912f7b51bf09f6b8953fc80285d60040cc98ddad14: Status 404 returned error can't find the container with id dcfb7b3dc43a797336d557912f7b51bf09f6b8953fc80285d60040cc98ddad14
Nov 02 12:01:21.496734 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.496677    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw"]
Nov 02 12:01:21.499332 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:01:21.499293    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf95cad35_61f5_4419_9533_c2f9add0d73e.slice/crio-e47d3dc82d8dcc3b63e8e73f7a617c71e01fe4470daa91db0150cae06e50adab WatchSource:0}: Error finding container e47d3dc82d8dcc3b63e8e73f7a617c71e01fe4470daa91db0150cae06e50adab: Status 404 returned error can't find the container with id e47d3dc82d8dcc3b63e8e73f7a617c71e01fe4470daa91db0150cae06e50adab
Nov 02 12:01:21.596957 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.596852    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c"]
Nov 02 12:01:21.599497 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:01:21.599475    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c00485_1170_4977_919a_8d463b407845.slice/crio-2bcf34a33c2a33be478ea2d2adc22f65cd2d6aa7c84fbc914013985c24b6b3d5 WatchSource:0}: Error finding container 2bcf34a33c2a33be478ea2d2adc22f65cd2d6aa7c84fbc914013985c24b6b3d5: Status 404 returned error can't find the container with id 2bcf34a33c2a33be478ea2d2adc22f65cd2d6aa7c84fbc914013985c24b6b3d5
Nov 02 12:01:21.692211 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.692186    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c" event={"ID":"53c00485-1170-4977-919a-8d463b407845","Type":"ContainerStarted","Data":"2bcf34a33c2a33be478ea2d2adc22f65cd2d6aa7c84fbc914013985c24b6b3d5"}
Nov 02 12:01:21.693399 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.693375    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw" event={"ID":"f95cad35-61f5-4419-9533-c2f9add0d73e","Type":"ContainerStarted","Data":"e47d3dc82d8dcc3b63e8e73f7a617c71e01fe4470daa91db0150cae06e50adab"}
Nov 02 12:01:21.694481 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.694463    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5" event={"ID":"a322fc87-6c9f-40a4-9d90-90a370bcdb01","Type":"ContainerStarted","Data":"bfa5069ff79dfe799546dfc5318b42a59483a751c128f3155e01290f35f37253"}
Nov 02 12:01:21.695487 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.695466    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk" event={"ID":"cca579e9-688a-4e73-8682-1a30b418ad54","Type":"ContainerStarted","Data":"8fdcd81b8e79bc0b692cde70841cece1d896cda91145fba767b2e6036d4c2bd2"}
Nov 02 12:01:21.696417 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:21.696399    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm" event={"ID":"b556fe62-b13c-4c41-b452-9543d5dc01d7","Type":"ContainerStarted","Data":"dcfb7b3dc43a797336d557912f7b51bf09f6b8953fc80285d60040cc98ddad14"}
Nov 02 12:01:22.995186 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:22.995154    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs"]
Nov 02 12:01:22.999812 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:22.999785    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs"
Nov 02 12:01:23.106168 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:23.106129    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs"]
Nov 02 12:01:23.106517 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:23.106491    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-xdj85\""
Nov 02 12:01:23.175088 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:23.174817    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4q6x\" (UniqueName: \"kubernetes.io/projected/590a79d8-8970-45c0-9cd5-a5a5710ca91a-kube-api-access-x4q6x\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs\" (UID: \"590a79d8-8970-45c0-9cd5-a5a5710ca91a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs"
Nov 02 12:01:23.175088 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:23.174883    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/590a79d8-8970-45c0-9cd5-a5a5710ca91a-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs\" (UID: \"590a79d8-8970-45c0-9cd5-a5a5710ca91a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs"
Nov 02 12:01:23.175088 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:23.174925    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/590a79d8-8970-45c0-9cd5-a5a5710ca91a-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs\" (UID: \"590a79d8-8970-45c0-9cd5-a5a5710ca91a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs"
Nov 02 12:01:23.278383 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:23.275998    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/590a79d8-8970-45c0-9cd5-a5a5710ca91a-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs\" (UID: \"590a79d8-8970-45c0-9cd5-a5a5710ca91a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs"
Nov 02 12:01:23.278383 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:23.276264    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4q6x\" (UniqueName: \"kubernetes.io/projected/590a79d8-8970-45c0-9cd5-a5a5710ca91a-kube-api-access-x4q6x\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs\" (UID: \"590a79d8-8970-45c0-9cd5-a5a5710ca91a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs"
Nov 02 12:01:23.278383 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:23.276343    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/590a79d8-8970-45c0-9cd5-a5a5710ca91a-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs\" (UID: \"590a79d8-8970-45c0-9cd5-a5a5710ca91a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs"
Nov 02 12:01:23.279424 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:23.279016    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/590a79d8-8970-45c0-9cd5-a5a5710ca91a-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs\" (UID: \"590a79d8-8970-45c0-9cd5-a5a5710ca91a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs"
Nov 02 12:01:23.280356 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:23.279817    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/590a79d8-8970-45c0-9cd5-a5a5710ca91a-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs\" (UID: \"590a79d8-8970-45c0-9cd5-a5a5710ca91a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs"
Nov 02 12:01:23.343100 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:23.343067    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4q6x\" (UniqueName: \"kubernetes.io/projected/590a79d8-8970-45c0-9cd5-a5a5710ca91a-kube-api-access-x4q6x\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs\" (UID: \"590a79d8-8970-45c0-9cd5-a5a5710ca91a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs"
Nov 02 12:01:23.630738 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:23.630704    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs"
Nov 02 12:01:24.045369 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:24.044986    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs"]
Nov 02 12:01:24.050141 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:01:24.050112    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod590a79d8_8970_45c0_9cd5_a5a5710ca91a.slice/crio-bdfcac3113daf87840fcb40c01018231e98e3920505d64f03182656231a4a3c2 WatchSource:0}: Error finding container bdfcac3113daf87840fcb40c01018231e98e3920505d64f03182656231a4a3c2: Status 404 returned error can't find the container with id bdfcac3113daf87840fcb40c01018231e98e3920505d64f03182656231a4a3c2
Nov 02 12:01:24.053215 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:24.052759    4942 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 12:01:24.347189 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:24.347110    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4"]
Nov 02 12:01:24.352064 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:24.351621    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4"
Nov 02 12:01:24.406338 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:24.406248    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4"]
Nov 02 12:01:24.453381 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:24.453351    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-x42jj\""
Nov 02 12:01:24.485995 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:24.485962    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4\" (UID: \"8722ab06-1b04-4396-9e36-2c4ba23c9bc5\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4"
Nov 02 12:01:24.486160 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:24.486031    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4\" (UID: \"8722ab06-1b04-4396-9e36-2c4ba23c9bc5\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4"
Nov 02 12:01:24.486160 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:24.486120    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-856pk\" (UniqueName: \"kubernetes.io/projected/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-kube-api-access-856pk\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4\" (UID: \"8722ab06-1b04-4396-9e36-2c4ba23c9bc5\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4"
Nov 02 12:01:24.586795 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:24.586756    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-856pk\" (UniqueName: \"kubernetes.io/projected/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-kube-api-access-856pk\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4\" (UID: \"8722ab06-1b04-4396-9e36-2c4ba23c9bc5\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4"
Nov 02 12:01:24.586977 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:24.586827    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4\" (UID: \"8722ab06-1b04-4396-9e36-2c4ba23c9bc5\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4"
Nov 02 12:01:24.586977 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:24.586868    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4\" (UID: \"8722ab06-1b04-4396-9e36-2c4ba23c9bc5\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4"
Nov 02 12:01:24.587282 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:24.587265    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4\" (UID: \"8722ab06-1b04-4396-9e36-2c4ba23c9bc5\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4"
Nov 02 12:01:24.587608 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:24.587585    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4\" (UID: \"8722ab06-1b04-4396-9e36-2c4ba23c9bc5\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4"
Nov 02 12:01:24.652738 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:24.652685    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-856pk\" (UniqueName: \"kubernetes.io/projected/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-kube-api-access-856pk\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4\" (UID: \"8722ab06-1b04-4396-9e36-2c4ba23c9bc5\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4"
Nov 02 12:01:24.674342 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:24.674312    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4"
Nov 02 12:01:24.726136 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:24.726096    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs" event={"ID":"590a79d8-8970-45c0-9cd5-a5a5710ca91a","Type":"ContainerStarted","Data":"bdfcac3113daf87840fcb40c01018231e98e3920505d64f03182656231a4a3c2"}
Nov 02 12:01:25.079988 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.079910    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4"]
Nov 02 12:01:25.090097 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:01:25.090063    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8722ab06_1b04_4396_9e36_2c4ba23c9bc5.slice/crio-3a4326d9b1a0c90556e6beb3e4d4bfb69c8f7afc19308e64904edee60647a6f5 WatchSource:0}: Error finding container 3a4326d9b1a0c90556e6beb3e4d4bfb69c8f7afc19308e64904edee60647a6f5: Status 404 returned error can't find the container with id 3a4326d9b1a0c90556e6beb3e4d4bfb69c8f7afc19308e64904edee60647a6f5
Nov 02 12:01:25.645879 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.645846    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"]
Nov 02 12:01:25.649985 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.649957    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.701719 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.700785    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.701719 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.700836    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnzr6\" (UniqueName: \"kubernetes.io/projected/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-kube-api-access-gnzr6\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.701719 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.700870    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.701719 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.700901    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.701719 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.700937    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.701719 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.700963    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.701719 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.701025    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.701719 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.701058    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.701719 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.701085    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.702523 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.702498    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 02 12:01:25.723452 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.723397    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-c5ldd\""
Nov 02 12:01:25.735394 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.735308    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4" event={"ID":"8722ab06-1b04-4396-9e36-2c4ba23c9bc5","Type":"ContainerStarted","Data":"3a4326d9b1a0c90556e6beb3e4d4bfb69c8f7afc19308e64904edee60647a6f5"}
Nov 02 12:01:25.756184 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.754003    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"]
Nov 02 12:01:25.801613 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.801543    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.801781 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.801634    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.801781 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.801669    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.801781 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.801730    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.801781 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.801772    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnzr6\" (UniqueName: \"kubernetes.io/projected/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-kube-api-access-gnzr6\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.801985 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.801805    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.801985 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.801832    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.801985 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.801875    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.801985 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.801903    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.802752 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.802703    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.803642 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.803616    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.803894 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.803864    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.804444 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.804418    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.805029 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.804935    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.805591 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.805566    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.812869 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.812847    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.863544 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.863482    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.867720 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.867670    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnzr6\" (UniqueName: \"kubernetes.io/projected/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-kube-api-access-gnzr6\") pod \"maas-default-gateway-openshift-default-6dcf757899-mg624\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.970067 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.969761    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:25.976974 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.976945    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"]
Nov 02 12:01:25.980673 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:25.980646    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.003999 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.003971    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/284e2d6f-6e51-4053-bf44-1809511df1f5-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.004115 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.004014    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/284e2d6f-6e51-4053-bf44-1809511df1f5-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.004115 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.004039    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/284e2d6f-6e51-4053-bf44-1809511df1f5-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.004115 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.004075    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/284e2d6f-6e51-4053-bf44-1809511df1f5-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.004115 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.004102    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/284e2d6f-6e51-4053-bf44-1809511df1f5-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.004336 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.004130    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/284e2d6f-6e51-4053-bf44-1809511df1f5-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.004336 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.004205    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgv2n\" (UniqueName: \"kubernetes.io/projected/284e2d6f-6e51-4053-bf44-1809511df1f5-kube-api-access-xgv2n\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.004336 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.004258    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/284e2d6f-6e51-4053-bf44-1809511df1f5-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.004336 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.004285    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/284e2d6f-6e51-4053-bf44-1809511df1f5-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.041786 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.041749    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"]
Nov 02 12:01:26.056145 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.056104    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-8pl4x\""
Nov 02 12:01:26.106014 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.105942    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgv2n\" (UniqueName: \"kubernetes.io/projected/284e2d6f-6e51-4053-bf44-1809511df1f5-kube-api-access-xgv2n\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.107680 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.106823    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/284e2d6f-6e51-4053-bf44-1809511df1f5-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.107680 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.106877    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/284e2d6f-6e51-4053-bf44-1809511df1f5-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.108238 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.108193    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/284e2d6f-6e51-4053-bf44-1809511df1f5-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.111046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.108406    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/284e2d6f-6e51-4053-bf44-1809511df1f5-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.111046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.108457    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/284e2d6f-6e51-4053-bf44-1809511df1f5-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.111046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.108483    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/284e2d6f-6e51-4053-bf44-1809511df1f5-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.111046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.108521    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/284e2d6f-6e51-4053-bf44-1809511df1f5-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.111046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.108549    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/284e2d6f-6e51-4053-bf44-1809511df1f5-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.111046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.108577    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/284e2d6f-6e51-4053-bf44-1809511df1f5-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.111046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.109746    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/284e2d6f-6e51-4053-bf44-1809511df1f5-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.111046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.110003    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/284e2d6f-6e51-4053-bf44-1809511df1f5-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.111046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.110474    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/284e2d6f-6e51-4053-bf44-1809511df1f5-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.111046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.110707    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/284e2d6f-6e51-4053-bf44-1809511df1f5-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.113927 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.113849    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/284e2d6f-6e51-4053-bf44-1809511df1f5-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.117061 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.117004    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/284e2d6f-6e51-4053-bf44-1809511df1f5-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.185273 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.185094    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgv2n\" (UniqueName: \"kubernetes.io/projected/284e2d6f-6e51-4053-bf44-1809511df1f5-kube-api-access-xgv2n\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.188908 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.188817    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/284e2d6f-6e51-4053-bf44-1809511df1f5-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk\" (UID: \"284e2d6f-6e51-4053-bf44-1809511df1f5\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.308154 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.307726    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:26.441361 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.441330    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"]
Nov 02 12:01:26.444527 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:01:26.444446    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c5fe5c5_f611_41ea_bc3f_f331df0c2c42.slice/crio-d0a700b3b48f788bc6250f48ff166f5c41bdda9fe4c6094834df5ff60f8b70f2 WatchSource:0}: Error finding container d0a700b3b48f788bc6250f48ff166f5c41bdda9fe4c6094834df5ff60f8b70f2: Status 404 returned error can't find the container with id d0a700b3b48f788bc6250f48ff166f5c41bdda9fe4c6094834df5ff60f8b70f2
Nov 02 12:01:26.749318 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.749256    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"]
Nov 02 12:01:26.755272 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:01:26.755238    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod284e2d6f_6e51_4053_bf44_1809511df1f5.slice/crio-a4a191828771f13f48ff313cfe52773ad27b4608a1e02bb782eeecea6668195c WatchSource:0}: Error finding container a4a191828771f13f48ff313cfe52773ad27b4608a1e02bb782eeecea6668195c: Status 404 returned error can't find the container with id a4a191828771f13f48ff313cfe52773ad27b4608a1e02bb782eeecea6668195c
Nov 02 12:01:26.767780 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:26.767746    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624" event={"ID":"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42","Type":"ContainerStarted","Data":"d0a700b3b48f788bc6250f48ff166f5c41bdda9fe4c6094834df5ff60f8b70f2"}
Nov 02 12:01:27.789957 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:27.789834    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk" event={"ID":"284e2d6f-6e51-4053-bf44-1809511df1f5","Type":"ContainerStarted","Data":"a4a191828771f13f48ff313cfe52773ad27b4608a1e02bb782eeecea6668195c"}
Nov 02 12:01:31.022788 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:01:31.022746    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:01:35.665276 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:35.665238    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:01:35.665598 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:35.665326    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:01:35.665598 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:35.665358    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:01:35.690000 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:35.689962    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:01:35.690098 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:35.690058    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:01:35.690098 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:35.690093    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:01:35.895047 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:35.895003    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624" event={"ID":"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42","Type":"ContainerStarted","Data":"de2beb053e81c92e915186a39876f11c76d59c750d59ce755b3f3a9143a8ebf4"}
Nov 02 12:01:35.905916 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:35.905325    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk" event={"ID":"284e2d6f-6e51-4053-bf44-1809511df1f5","Type":"ContainerStarted","Data":"0729ffe077970702d63537eeacbd6f3c2fba4d12f2ced40be226165fa29da504"}
Nov 02 12:01:35.970557 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:35.970438    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:36.059424 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:36.059365    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624" podStartSLOduration=1.8204902299999999 podStartE2EDuration="11.059345135s" totalImagesPullingTime="9.238854905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:01:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:01:36.057729913 +0000 UTC m=+322.629079674" watchObservedRunningTime="2026-11-02 12:01:36.059345135 +0000 UTC m=+322.630694894"
Nov 02 12:01:36.179907 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:36.179843    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk" podStartSLOduration=2.284544528 podStartE2EDuration="11.179822757s" totalImagesPullingTime="8.895278229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:01:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:01:36.179234184 +0000 UTC m=+322.750583942" watchObservedRunningTime="2026-11-02 12:01:36.179822757 +0000 UTC m=+322.751172518"
Nov 02 12:01:36.309021 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:36.308934    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:36.324011 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:36.323739    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:36.911627 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:36.911563    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:36.912878 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:36.912672    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk"
Nov 02 12:01:36.976315 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:36.976240    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:37.916991 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:37.916954    4942 generic.go:356] "Generic (PLEG): container finished" podID="b556fe62-b13c-4c41-b452-9543d5dc01d7" containerID="c8c9b5ff304ce4e1d0710e00b04a30c446dc59e14ea9b5ff18f51d7892893065" exitCode=0
Nov 02 12:01:37.917472 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:37.917034    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm" event={"ID":"b556fe62-b13c-4c41-b452-9543d5dc01d7","Type":"ContainerDied","Data":"c8c9b5ff304ce4e1d0710e00b04a30c446dc59e14ea9b5ff18f51d7892893065"}
Nov 02 12:01:37.919481 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:37.919455    4942 generic.go:356] "Generic (PLEG): container finished" podID="53c00485-1170-4977-919a-8d463b407845" containerID="e4bdfbd125df0931e2b1ed1f99e73db8b5660f0fc84e1a8f89e269b2b624d687" exitCode=0
Nov 02 12:01:37.919576 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:37.919536    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c" event={"ID":"53c00485-1170-4977-919a-8d463b407845","Type":"ContainerDied","Data":"e4bdfbd125df0931e2b1ed1f99e73db8b5660f0fc84e1a8f89e269b2b624d687"}
Nov 02 12:01:37.921837 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:37.921813    4942 generic.go:356] "Generic (PLEG): container finished" podID="f95cad35-61f5-4419-9533-c2f9add0d73e" containerID="8d0a2ff8c780f008cb5c379c0cf4dc2df77cb7216800a53f4eee6774e3fe31c6" exitCode=0
Nov 02 12:01:37.921940 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:37.921897    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw" event={"ID":"f95cad35-61f5-4419-9533-c2f9add0d73e","Type":"ContainerDied","Data":"8d0a2ff8c780f008cb5c379c0cf4dc2df77cb7216800a53f4eee6774e3fe31c6"}
Nov 02 12:01:37.924353 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:37.924329    4942 generic.go:356] "Generic (PLEG): container finished" podID="a322fc87-6c9f-40a4-9d90-90a370bcdb01" containerID="c3d907123845c8d60497235abaa6dbb47eb0b812e7d25c8d11b59a9c3bd25226" exitCode=0
Nov 02 12:01:37.924458 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:37.924413    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5" event={"ID":"a322fc87-6c9f-40a4-9d90-90a370bcdb01","Type":"ContainerDied","Data":"c3d907123845c8d60497235abaa6dbb47eb0b812e7d25c8d11b59a9c3bd25226"}
Nov 02 12:01:37.926494 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:37.926257    4942 generic.go:356] "Generic (PLEG): container finished" podID="cca579e9-688a-4e73-8682-1a30b418ad54" containerID="cb10be468893f5de720dff2818bd12165d9401b438c7975fc2558d5bdd7e565d" exitCode=0
Nov 02 12:01:37.926494 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:37.926333    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk" event={"ID":"cca579e9-688a-4e73-8682-1a30b418ad54","Type":"ContainerDied","Data":"cb10be468893f5de720dff2818bd12165d9401b438c7975fc2558d5bdd7e565d"}
Nov 02 12:01:37.926929 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:37.926906    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:37.927765 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:37.927747    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:01:38.933861 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:38.933823    4942 generic.go:356] "Generic (PLEG): container finished" podID="590a79d8-8970-45c0-9cd5-a5a5710ca91a" containerID="f4c7a189e7db4c51ae6dec62aa6427267dfd74ee67593eaa024364bc068f289c" exitCode=0
Nov 02 12:01:38.934274 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:38.933914    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs" event={"ID":"590a79d8-8970-45c0-9cd5-a5a5710ca91a","Type":"ContainerDied","Data":"f4c7a189e7db4c51ae6dec62aa6427267dfd74ee67593eaa024364bc068f289c"}
Nov 02 12:01:39.940942 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:39.940908    4942 generic.go:356] "Generic (PLEG): container finished" podID="8722ab06-1b04-4396-9e36-2c4ba23c9bc5" containerID="6eec24a0076e5a62793c970604160899cf32ec4e4e43ad75d1b27e37e7c88c0f" exitCode=0
Nov 02 12:01:39.941348 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:39.941006    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4" event={"ID":"8722ab06-1b04-4396-9e36-2c4ba23c9bc5","Type":"ContainerDied","Data":"6eec24a0076e5a62793c970604160899cf32ec4e4e43ad75d1b27e37e7c88c0f"}
Nov 02 12:01:39.943470 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:39.943446    4942 generic.go:356] "Generic (PLEG): container finished" podID="cca579e9-688a-4e73-8682-1a30b418ad54" containerID="31ea23f798b5c454f9380d811231c21e123d74c3ce7fa510a04035f29bc8e3f5" exitCode=0
Nov 02 12:01:39.943581 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:39.943511    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk" event={"ID":"cca579e9-688a-4e73-8682-1a30b418ad54","Type":"ContainerDied","Data":"31ea23f798b5c454f9380d811231c21e123d74c3ce7fa510a04035f29bc8e3f5"}
Nov 02 12:01:39.945524 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:39.945493    4942 generic.go:356] "Generic (PLEG): container finished" podID="b556fe62-b13c-4c41-b452-9543d5dc01d7" containerID="33f59b3795e998c1252218f20225ef496035fe1034e0db2a8bc49dee3a65d728" exitCode=0
Nov 02 12:01:39.945630 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:39.945614    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm" event={"ID":"b556fe62-b13c-4c41-b452-9543d5dc01d7","Type":"ContainerDied","Data":"33f59b3795e998c1252218f20225ef496035fe1034e0db2a8bc49dee3a65d728"}
Nov 02 12:01:39.947598 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:39.947576    4942 generic.go:356] "Generic (PLEG): container finished" podID="53c00485-1170-4977-919a-8d463b407845" containerID="4db2a80563346cbd390ead6f36a2e16ea00beb41d8dc8603bb42a04d5486b88f" exitCode=0
Nov 02 12:01:39.947689 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:39.947664    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c" event={"ID":"53c00485-1170-4977-919a-8d463b407845","Type":"ContainerDied","Data":"4db2a80563346cbd390ead6f36a2e16ea00beb41d8dc8603bb42a04d5486b88f"}
Nov 02 12:01:39.949794 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:39.949724    4942 generic.go:356] "Generic (PLEG): container finished" podID="f95cad35-61f5-4419-9533-c2f9add0d73e" containerID="72b47db4d66ecb88e47a56b27e5d916720dfe582958f5a35ed3b6c3f3063b447" exitCode=0
Nov 02 12:01:39.949912 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:39.949889    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw" event={"ID":"f95cad35-61f5-4419-9533-c2f9add0d73e","Type":"ContainerDied","Data":"72b47db4d66ecb88e47a56b27e5d916720dfe582958f5a35ed3b6c3f3063b447"}
Nov 02 12:01:40.963882 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:40.963840    4942 generic.go:356] "Generic (PLEG): container finished" podID="8722ab06-1b04-4396-9e36-2c4ba23c9bc5" containerID="2e0eb050c118cd3ecec32165e4658579faa15374472a79759fbaf3ef1fca2ea9" exitCode=0
Nov 02 12:01:40.964354 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:40.963896    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4" event={"ID":"8722ab06-1b04-4396-9e36-2c4ba23c9bc5","Type":"ContainerDied","Data":"2e0eb050c118cd3ecec32165e4658579faa15374472a79759fbaf3ef1fca2ea9"}
Nov 02 12:01:41.974471 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:41.974056    4942 generic.go:356] "Generic (PLEG): container finished" podID="a322fc87-6c9f-40a4-9d90-90a370bcdb01" containerID="e02e0828acea5abf0f11f436080f52a98312e993e4375e990b7225f51bd480ca" exitCode=0
Nov 02 12:01:41.974471 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:41.974189    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5" event={"ID":"a322fc87-6c9f-40a4-9d90-90a370bcdb01","Type":"ContainerDied","Data":"e02e0828acea5abf0f11f436080f52a98312e993e4375e990b7225f51bd480ca"}
Nov 02 12:01:43.997332 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:43.997229    4942 generic.go:356] "Generic (PLEG): container finished" podID="590a79d8-8970-45c0-9cd5-a5a5710ca91a" containerID="a20ad445bda40371e3dc88e13f2121058a584e343031cec0b1d825d1aa4f9096" exitCode=0
Nov 02 12:01:43.998237 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:43.998199    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs" event={"ID":"590a79d8-8970-45c0-9cd5-a5a5710ca91a","Type":"ContainerDied","Data":"a20ad445bda40371e3dc88e13f2121058a584e343031cec0b1d825d1aa4f9096"}
Nov 02 12:01:55.105036 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:55.105002    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk" event={"ID":"cca579e9-688a-4e73-8682-1a30b418ad54","Type":"ContainerStarted","Data":"d63cc288c5c7ddcdea58112e01051a9c43a6c585600ef1720b3555793934eb82"}
Nov 02 12:01:55.108107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:55.107715    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm" event={"ID":"b556fe62-b13c-4c41-b452-9543d5dc01d7","Type":"ContainerStarted","Data":"bc6ac5aa242796e23b9c3a5c3b234d8ade97e07d8030d18ab4de3ce7d136a4e7"}
Nov 02 12:01:55.110247 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:55.110221    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c" event={"ID":"53c00485-1170-4977-919a-8d463b407845","Type":"ContainerStarted","Data":"84813018fde790e839639905d707d898bc8daed6558f587adb074addc4591500"}
Nov 02 12:01:55.112612 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:55.112587    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw" event={"ID":"f95cad35-61f5-4419-9533-c2f9add0d73e","Type":"ContainerStarted","Data":"2e842bbf64755c1a12c2cc29295f3181b02bb449f6b7e724facde7b4f48a832a"}
Nov 02 12:01:55.264241 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:55.262695    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk" podStartSLOduration=3.415269972 podStartE2EDuration="35.262682195s" totalImagesPullingTime="31.847412223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:01:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:01:55.254690355 +0000 UTC m=+341.826040109" watchObservedRunningTime="2026-11-02 12:01:55.262682195 +0000 UTC m=+341.834031952"
Nov 02 12:01:55.375099 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:55.374797    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw" podStartSLOduration=2.443659281 podStartE2EDuration="35.37478068s" totalImagesPullingTime="32.931121399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:01:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:01:55.372239896 +0000 UTC m=+341.943589665" watchObservedRunningTime="2026-11-02 12:01:55.37478068 +0000 UTC m=+341.946130438"
Nov 02 12:01:55.491152 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:55.491100    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c" podStartSLOduration=2.920238604 podStartE2EDuration="35.491085913s" totalImagesPullingTime="32.570847309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:01:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:01:55.489266453 +0000 UTC m=+342.060616209" watchObservedRunningTime="2026-11-02 12:01:55.491085913 +0000 UTC m=+342.062435667"
Nov 02 12:01:55.608354 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:55.608291    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm" podStartSLOduration=3.097446711 podStartE2EDuration="35.608271095s" totalImagesPullingTime="32.510824384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:01:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:01:55.606863603 +0000 UTC m=+342.178213363" watchObservedRunningTime="2026-11-02 12:01:55.608271095 +0000 UTC m=+342.179620896"
Nov 02 12:01:56.120172 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:56.120133    4942 generic.go:356] "Generic (PLEG): container finished" podID="8722ab06-1b04-4396-9e36-2c4ba23c9bc5" containerID="144c2f290f62962f74a99ec9de55e94348baea8060f07fecb9caa0f9e19d099d" exitCode=0
Nov 02 12:01:56.120676 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:56.120573    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4" event={"ID":"8722ab06-1b04-4396-9e36-2c4ba23c9bc5","Type":"ContainerDied","Data":"144c2f290f62962f74a99ec9de55e94348baea8060f07fecb9caa0f9e19d099d"}
Nov 02 12:01:56.123129 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:56.123097    4942 generic.go:356] "Generic (PLEG): container finished" podID="cca579e9-688a-4e73-8682-1a30b418ad54" containerID="d63cc288c5c7ddcdea58112e01051a9c43a6c585600ef1720b3555793934eb82" exitCode=0
Nov 02 12:01:56.123325 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:56.123194    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk" event={"ID":"cca579e9-688a-4e73-8682-1a30b418ad54","Type":"ContainerDied","Data":"d63cc288c5c7ddcdea58112e01051a9c43a6c585600ef1720b3555793934eb82"}
Nov 02 12:01:56.126127 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:56.126098    4942 generic.go:356] "Generic (PLEG): container finished" podID="b556fe62-b13c-4c41-b452-9543d5dc01d7" containerID="bc6ac5aa242796e23b9c3a5c3b234d8ade97e07d8030d18ab4de3ce7d136a4e7" exitCode=0
Nov 02 12:01:56.133376 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:56.133343    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm" event={"ID":"b556fe62-b13c-4c41-b452-9543d5dc01d7","Type":"ContainerDied","Data":"bc6ac5aa242796e23b9c3a5c3b234d8ade97e07d8030d18ab4de3ce7d136a4e7"}
Nov 02 12:01:56.138587 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:56.138559    4942 generic.go:356] "Generic (PLEG): container finished" podID="53c00485-1170-4977-919a-8d463b407845" containerID="84813018fde790e839639905d707d898bc8daed6558f587adb074addc4591500" exitCode=0
Nov 02 12:01:56.139496 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:56.138692    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c" event={"ID":"53c00485-1170-4977-919a-8d463b407845","Type":"ContainerDied","Data":"84813018fde790e839639905d707d898bc8daed6558f587adb074addc4591500"}
Nov 02 12:01:56.141874 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:56.141773    4942 generic.go:356] "Generic (PLEG): container finished" podID="f95cad35-61f5-4419-9533-c2f9add0d73e" containerID="2e842bbf64755c1a12c2cc29295f3181b02bb449f6b7e724facde7b4f48a832a" exitCode=0
Nov 02 12:01:56.141874 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:56.141843    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw" event={"ID":"f95cad35-61f5-4419-9533-c2f9add0d73e","Type":"ContainerDied","Data":"2e842bbf64755c1a12c2cc29295f3181b02bb449f6b7e724facde7b4f48a832a"}
Nov 02 12:01:57.148907 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.148869    4942 generic.go:356] "Generic (PLEG): container finished" podID="a322fc87-6c9f-40a4-9d90-90a370bcdb01" containerID="0f105118fd90d284d35ee8fdfda75836a8e5c72f965a1f784ef1490c5a85436f" exitCode=0
Nov 02 12:01:57.149277 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.148914    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5" event={"ID":"a322fc87-6c9f-40a4-9d90-90a370bcdb01","Type":"ContainerDied","Data":"0f105118fd90d284d35ee8fdfda75836a8e5c72f965a1f784ef1490c5a85436f"}
Nov 02 12:01:57.338263 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.334393    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk"
Nov 02 12:01:57.358108 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.358085    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw"
Nov 02 12:01:57.379949 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.379930    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm"
Nov 02 12:01:57.383110 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.383088    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c"
Nov 02 12:01:57.387343 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.387323    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4"
Nov 02 12:01:57.439800 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.439776    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f95cad35-61f5-4419-9533-c2f9add0d73e-bundle\" (UniqueName: \"kubernetes.io/empty-dir/f95cad35-61f5-4419-9533-c2f9add0d73e-bundle\") pod \"f95cad35-61f5-4419-9533-c2f9add0d73e\" (UID: \"f95cad35-61f5-4419-9533-c2f9add0d73e\") "
Nov 02 12:01:57.439934 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.439860    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f95cad35-61f5-4419-9533-c2f9add0d73e-util\" (UniqueName: \"kubernetes.io/empty-dir/f95cad35-61f5-4419-9533-c2f9add0d73e-util\") pod \"f95cad35-61f5-4419-9533-c2f9add0d73e\" (UID: \"f95cad35-61f5-4419-9533-c2f9add0d73e\") "
Nov 02 12:01:57.439934 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.439893    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cca579e9-688a-4e73-8682-1a30b418ad54-util\" (UniqueName: \"kubernetes.io/empty-dir/cca579e9-688a-4e73-8682-1a30b418ad54-util\") pod \"cca579e9-688a-4e73-8682-1a30b418ad54\" (UID: \"cca579e9-688a-4e73-8682-1a30b418ad54\") "
Nov 02 12:01:57.439934 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.439922    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cca579e9-688a-4e73-8682-1a30b418ad54-kube-api-access-4ks7c\" (UniqueName: \"kubernetes.io/projected/cca579e9-688a-4e73-8682-1a30b418ad54-kube-api-access-4ks7c\") pod \"cca579e9-688a-4e73-8682-1a30b418ad54\" (UID: \"cca579e9-688a-4e73-8682-1a30b418ad54\") "
Nov 02 12:01:57.440093 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.439956    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f95cad35-61f5-4419-9533-c2f9add0d73e-kube-api-access-2stdk\" (UniqueName: \"kubernetes.io/projected/f95cad35-61f5-4419-9533-c2f9add0d73e-kube-api-access-2stdk\") pod \"f95cad35-61f5-4419-9533-c2f9add0d73e\" (UID: \"f95cad35-61f5-4419-9533-c2f9add0d73e\") "
Nov 02 12:01:57.440093 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.439993    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cca579e9-688a-4e73-8682-1a30b418ad54-bundle\" (UniqueName: \"kubernetes.io/empty-dir/cca579e9-688a-4e73-8682-1a30b418ad54-bundle\") pod \"cca579e9-688a-4e73-8682-1a30b418ad54\" (UID: \"cca579e9-688a-4e73-8682-1a30b418ad54\") "
Nov 02 12:01:57.440480 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.440448    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95cad35-61f5-4419-9533-c2f9add0d73e-bundle" pod "f95cad35-61f5-4419-9533-c2f9add0d73e" (UID: "f95cad35-61f5-4419-9533-c2f9add0d73e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:01:57.440652 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.440610    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cca579e9-688a-4e73-8682-1a30b418ad54-bundle" pod "cca579e9-688a-4e73-8682-1a30b418ad54" (UID: "cca579e9-688a-4e73-8682-1a30b418ad54"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:01:57.442232 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.442210    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cca579e9-688a-4e73-8682-1a30b418ad54-kube-api-access-4ks7c" pod "cca579e9-688a-4e73-8682-1a30b418ad54" (UID: "cca579e9-688a-4e73-8682-1a30b418ad54"). InnerVolumeSpecName "kube-api-access-4ks7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:01:57.442415 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.442263    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f95cad35-61f5-4419-9533-c2f9add0d73e-kube-api-access-2stdk" pod "f95cad35-61f5-4419-9533-c2f9add0d73e" (UID: "f95cad35-61f5-4419-9533-c2f9add0d73e"). InnerVolumeSpecName "kube-api-access-2stdk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:01:57.444209 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.444190    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cca579e9-688a-4e73-8682-1a30b418ad54-util" pod "cca579e9-688a-4e73-8682-1a30b418ad54" (UID: "cca579e9-688a-4e73-8682-1a30b418ad54"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:01:57.445944 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.445923    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95cad35-61f5-4419-9533-c2f9add0d73e-util" pod "f95cad35-61f5-4419-9533-c2f9add0d73e" (UID: "f95cad35-61f5-4419-9533-c2f9add0d73e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:01:57.541009 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.540982    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-bundle\" (UniqueName: \"kubernetes.io/empty-dir/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-bundle\") pod \"8722ab06-1b04-4396-9e36-2c4ba23c9bc5\" (UID: \"8722ab06-1b04-4396-9e36-2c4ba23c9bc5\") "
Nov 02 12:01:57.541167 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.541014    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53c00485-1170-4977-919a-8d463b407845-util\" (UniqueName: \"kubernetes.io/empty-dir/53c00485-1170-4977-919a-8d463b407845-util\") pod \"53c00485-1170-4977-919a-8d463b407845\" (UID: \"53c00485-1170-4977-919a-8d463b407845\") "
Nov 02 12:01:57.541167 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.541047    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-util\" (UniqueName: \"kubernetes.io/empty-dir/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-util\") pod \"8722ab06-1b04-4396-9e36-2c4ba23c9bc5\" (UID: \"8722ab06-1b04-4396-9e36-2c4ba23c9bc5\") "
Nov 02 12:01:57.541167 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.541083    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b556fe62-b13c-4c41-b452-9543d5dc01d7-kube-api-access-r2llm\" (UniqueName: \"kubernetes.io/projected/b556fe62-b13c-4c41-b452-9543d5dc01d7-kube-api-access-r2llm\") pod \"b556fe62-b13c-4c41-b452-9543d5dc01d7\" (UID: \"b556fe62-b13c-4c41-b452-9543d5dc01d7\") "
Nov 02 12:01:57.541167 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.541137    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53c00485-1170-4977-919a-8d463b407845-kube-api-access-h6w4x\" (UniqueName: \"kubernetes.io/projected/53c00485-1170-4977-919a-8d463b407845-kube-api-access-h6w4x\") pod \"53c00485-1170-4977-919a-8d463b407845\" (UID: \"53c00485-1170-4977-919a-8d463b407845\") "
Nov 02 12:01:57.541408 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.541169    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-kube-api-access-856pk\" (UniqueName: \"kubernetes.io/projected/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-kube-api-access-856pk\") pod \"8722ab06-1b04-4396-9e36-2c4ba23c9bc5\" (UID: \"8722ab06-1b04-4396-9e36-2c4ba23c9bc5\") "
Nov 02 12:01:57.541408 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.541234    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53c00485-1170-4977-919a-8d463b407845-bundle\" (UniqueName: \"kubernetes.io/empty-dir/53c00485-1170-4977-919a-8d463b407845-bundle\") pod \"53c00485-1170-4977-919a-8d463b407845\" (UID: \"53c00485-1170-4977-919a-8d463b407845\") "
Nov 02 12:01:57.541408 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.541387    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b556fe62-b13c-4c41-b452-9543d5dc01d7-bundle\" (UniqueName: \"kubernetes.io/empty-dir/b556fe62-b13c-4c41-b452-9543d5dc01d7-bundle\") pod \"b556fe62-b13c-4c41-b452-9543d5dc01d7\" (UID: \"b556fe62-b13c-4c41-b452-9543d5dc01d7\") "
Nov 02 12:01:57.541575 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.541450    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b556fe62-b13c-4c41-b452-9543d5dc01d7-util\" (UniqueName: \"kubernetes.io/empty-dir/b556fe62-b13c-4c41-b452-9543d5dc01d7-util\") pod \"b556fe62-b13c-4c41-b452-9543d5dc01d7\" (UID: \"b556fe62-b13c-4c41-b452-9543d5dc01d7\") "
Nov 02 12:01:57.541735 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.541711    4942 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f95cad35-61f5-4419-9533-c2f9add0d73e-bundle\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:57.541735 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.541736    4942 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f95cad35-61f5-4419-9533-c2f9add0d73e-util\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:57.541914 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.541748    4942 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cca579e9-688a-4e73-8682-1a30b418ad54-util\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:57.541914 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.541762    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ks7c\" (UniqueName: \"kubernetes.io/projected/cca579e9-688a-4e73-8682-1a30b418ad54-kube-api-access-4ks7c\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:57.541914 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.541777    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2stdk\" (UniqueName: \"kubernetes.io/projected/f95cad35-61f5-4419-9533-c2f9add0d73e-kube-api-access-2stdk\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:57.541914 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.541791    4942 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cca579e9-688a-4e73-8682-1a30b418ad54-bundle\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:57.541914 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.541840    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-bundle" pod "8722ab06-1b04-4396-9e36-2c4ba23c9bc5" (UID: "8722ab06-1b04-4396-9e36-2c4ba23c9bc5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:01:57.542582 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.542554    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b556fe62-b13c-4c41-b452-9543d5dc01d7-bundle" pod "b556fe62-b13c-4c41-b452-9543d5dc01d7" (UID: "b556fe62-b13c-4c41-b452-9543d5dc01d7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:01:57.542798 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.542778    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53c00485-1170-4977-919a-8d463b407845-bundle" pod "53c00485-1170-4977-919a-8d463b407845" (UID: "53c00485-1170-4977-919a-8d463b407845"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:01:57.544099 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.544070    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b556fe62-b13c-4c41-b452-9543d5dc01d7-kube-api-access-r2llm" pod "b556fe62-b13c-4c41-b452-9543d5dc01d7" (UID: "b556fe62-b13c-4c41-b452-9543d5dc01d7"). InnerVolumeSpecName "kube-api-access-r2llm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:01:57.545406 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.545377    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53c00485-1170-4977-919a-8d463b407845-kube-api-access-h6w4x" pod "53c00485-1170-4977-919a-8d463b407845" (UID: "53c00485-1170-4977-919a-8d463b407845"). InnerVolumeSpecName "kube-api-access-h6w4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:01:57.547118 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.547093    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53c00485-1170-4977-919a-8d463b407845-util" pod "53c00485-1170-4977-919a-8d463b407845" (UID: "53c00485-1170-4977-919a-8d463b407845"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:01:57.549795 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.549771    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b556fe62-b13c-4c41-b452-9543d5dc01d7-util" pod "b556fe62-b13c-4c41-b452-9543d5dc01d7" (UID: "b556fe62-b13c-4c41-b452-9543d5dc01d7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:01:57.550424 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.550404    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-util" pod "8722ab06-1b04-4396-9e36-2c4ba23c9bc5" (UID: "8722ab06-1b04-4396-9e36-2c4ba23c9bc5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:01:57.550873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.550821    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-kube-api-access-856pk" pod "8722ab06-1b04-4396-9e36-2c4ba23c9bc5" (UID: "8722ab06-1b04-4396-9e36-2c4ba23c9bc5"). InnerVolumeSpecName "kube-api-access-856pk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:01:57.642955 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.642923    4942 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53c00485-1170-4977-919a-8d463b407845-bundle\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:57.642955 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.642946    4942 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b556fe62-b13c-4c41-b452-9543d5dc01d7-bundle\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:57.643130 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.642959    4942 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b556fe62-b13c-4c41-b452-9543d5dc01d7-util\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:57.643130 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.642971    4942 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-bundle\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:57.643130 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.642981    4942 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53c00485-1170-4977-919a-8d463b407845-util\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:57.643130 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.642991    4942 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-util\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:57.643130 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.643003    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2llm\" (UniqueName: \"kubernetes.io/projected/b556fe62-b13c-4c41-b452-9543d5dc01d7-kube-api-access-r2llm\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:57.643130 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.643017    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6w4x\" (UniqueName: \"kubernetes.io/projected/53c00485-1170-4977-919a-8d463b407845-kube-api-access-h6w4x\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:57.643130 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:57.643034    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-856pk\" (UniqueName: \"kubernetes.io/projected/8722ab06-1b04-4396-9e36-2c4ba23c9bc5-kube-api-access-856pk\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:58.155169 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.155136    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm" event={"ID":"b556fe62-b13c-4c41-b452-9543d5dc01d7","Type":"ContainerDied","Data":"dcfb7b3dc43a797336d557912f7b51bf09f6b8953fc80285d60040cc98ddad14"}
Nov 02 12:01:58.155512 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.155195    4942 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dcfb7b3dc43a797336d557912f7b51bf09f6b8953fc80285d60040cc98ddad14"
Nov 02 12:01:58.155512 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.155145    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847azftxm"
Nov 02 12:01:58.157327 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.157285    4942 generic.go:356] "Generic (PLEG): container finished" podID="590a79d8-8970-45c0-9cd5-a5a5710ca91a" containerID="37d388020ba059834bd7333026a7d92631762267b23b83d0465176d696b670dc" exitCode=0
Nov 02 12:01:58.157420 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.157328    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs" event={"ID":"590a79d8-8970-45c0-9cd5-a5a5710ca91a","Type":"ContainerDied","Data":"37d388020ba059834bd7333026a7d92631762267b23b83d0465176d696b670dc"}
Nov 02 12:01:58.159229 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.159203    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c" event={"ID":"53c00485-1170-4977-919a-8d463b407845","Type":"ContainerDied","Data":"2bcf34a33c2a33be478ea2d2adc22f65cd2d6aa7c84fbc914013985c24b6b3d5"}
Nov 02 12:01:58.159229 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.159226    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6tbh8c"
Nov 02 12:01:58.159370 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.159246    4942 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2bcf34a33c2a33be478ea2d2adc22f65cd2d6aa7c84fbc914013985c24b6b3d5"
Nov 02 12:01:58.161140 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.161126    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw"
Nov 02 12:01:58.161234 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.161156    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czzzmw" event={"ID":"f95cad35-61f5-4419-9533-c2f9add0d73e","Type":"ContainerDied","Data":"e47d3dc82d8dcc3b63e8e73f7a617c71e01fe4470daa91db0150cae06e50adab"}
Nov 02 12:01:58.161234 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.161199    4942 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e47d3dc82d8dcc3b63e8e73f7a617c71e01fe4470daa91db0150cae06e50adab"
Nov 02 12:01:58.163480 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.163459    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4" event={"ID":"8722ab06-1b04-4396-9e36-2c4ba23c9bc5","Type":"ContainerDied","Data":"3a4326d9b1a0c90556e6beb3e4d4bfb69c8f7afc19308e64904edee60647a6f5"}
Nov 02 12:01:58.163580 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.163495    4942 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a4326d9b1a0c90556e6beb3e4d4bfb69c8f7afc19308e64904edee60647a6f5"
Nov 02 12:01:58.163580 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.163570    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71m8vv4"
Nov 02 12:01:58.166495 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.166471    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk" event={"ID":"cca579e9-688a-4e73-8682-1a30b418ad54","Type":"ContainerDied","Data":"8fdcd81b8e79bc0b692cde70841cece1d896cda91145fba767b2e6036d4c2bd2"}
Nov 02 12:01:58.166586 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.166515    4942 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8fdcd81b8e79bc0b692cde70841cece1d896cda91145fba767b2e6036d4c2bd2"
Nov 02 12:01:58.166648 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.166634    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7q6tsk"
Nov 02 12:01:58.287936 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.287911    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5"
Nov 02 12:01:58.449587 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.449551    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a322fc87-6c9f-40a4-9d90-90a370bcdb01-kube-api-access-mt9ck\" (UniqueName: \"kubernetes.io/projected/a322fc87-6c9f-40a4-9d90-90a370bcdb01-kube-api-access-mt9ck\") pod \"a322fc87-6c9f-40a4-9d90-90a370bcdb01\" (UID: \"a322fc87-6c9f-40a4-9d90-90a370bcdb01\") "
Nov 02 12:01:58.449740 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.449613    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a322fc87-6c9f-40a4-9d90-90a370bcdb01-util\" (UniqueName: \"kubernetes.io/empty-dir/a322fc87-6c9f-40a4-9d90-90a370bcdb01-util\") pod \"a322fc87-6c9f-40a4-9d90-90a370bcdb01\" (UID: \"a322fc87-6c9f-40a4-9d90-90a370bcdb01\") "
Nov 02 12:01:58.449740 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.449688    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a322fc87-6c9f-40a4-9d90-90a370bcdb01-bundle\" (UniqueName: \"kubernetes.io/empty-dir/a322fc87-6c9f-40a4-9d90-90a370bcdb01-bundle\") pod \"a322fc87-6c9f-40a4-9d90-90a370bcdb01\" (UID: \"a322fc87-6c9f-40a4-9d90-90a370bcdb01\") "
Nov 02 12:01:58.450059 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.450037    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a322fc87-6c9f-40a4-9d90-90a370bcdb01-bundle" pod "a322fc87-6c9f-40a4-9d90-90a370bcdb01" (UID: "a322fc87-6c9f-40a4-9d90-90a370bcdb01"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:01:58.451666 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.451642    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a322fc87-6c9f-40a4-9d90-90a370bcdb01-kube-api-access-mt9ck" pod "a322fc87-6c9f-40a4-9d90-90a370bcdb01" (UID: "a322fc87-6c9f-40a4-9d90-90a370bcdb01"). InnerVolumeSpecName "kube-api-access-mt9ck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:01:58.454209 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.454186    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a322fc87-6c9f-40a4-9d90-90a370bcdb01-util" pod "a322fc87-6c9f-40a4-9d90-90a370bcdb01" (UID: "a322fc87-6c9f-40a4-9d90-90a370bcdb01"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:01:58.551247 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.551175    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt9ck\" (UniqueName: \"kubernetes.io/projected/a322fc87-6c9f-40a4-9d90-90a370bcdb01-kube-api-access-mt9ck\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:58.551247 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.551202    4942 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a322fc87-6c9f-40a4-9d90-90a370bcdb01-util\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:58.551247 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:58.551213    4942 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a322fc87-6c9f-40a4-9d90-90a370bcdb01-bundle\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:59.176531 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:59.176500    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5"
Nov 02 12:01:59.176531 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:59.176515    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644j5zm5" event={"ID":"a322fc87-6c9f-40a4-9d90-90a370bcdb01","Type":"ContainerDied","Data":"bfa5069ff79dfe799546dfc5318b42a59483a751c128f3155e01290f35f37253"}
Nov 02 12:01:59.176974 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:59.176581    4942 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bfa5069ff79dfe799546dfc5318b42a59483a751c128f3155e01290f35f37253"
Nov 02 12:01:59.300358 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:59.300336    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs"
Nov 02 12:01:59.459639 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:59.459573    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/590a79d8-8970-45c0-9cd5-a5a5710ca91a-kube-api-access-x4q6x\" (UniqueName: \"kubernetes.io/projected/590a79d8-8970-45c0-9cd5-a5a5710ca91a-kube-api-access-x4q6x\") pod \"590a79d8-8970-45c0-9cd5-a5a5710ca91a\" (UID: \"590a79d8-8970-45c0-9cd5-a5a5710ca91a\") "
Nov 02 12:01:59.459639 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:59.459620    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/590a79d8-8970-45c0-9cd5-a5a5710ca91a-bundle\" (UniqueName: \"kubernetes.io/empty-dir/590a79d8-8970-45c0-9cd5-a5a5710ca91a-bundle\") pod \"590a79d8-8970-45c0-9cd5-a5a5710ca91a\" (UID: \"590a79d8-8970-45c0-9cd5-a5a5710ca91a\") "
Nov 02 12:01:59.459808 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:59.459649    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/590a79d8-8970-45c0-9cd5-a5a5710ca91a-util\" (UniqueName: \"kubernetes.io/empty-dir/590a79d8-8970-45c0-9cd5-a5a5710ca91a-util\") pod \"590a79d8-8970-45c0-9cd5-a5a5710ca91a\" (UID: \"590a79d8-8970-45c0-9cd5-a5a5710ca91a\") "
Nov 02 12:01:59.460420 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:59.460393    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/590a79d8-8970-45c0-9cd5-a5a5710ca91a-bundle" pod "590a79d8-8970-45c0-9cd5-a5a5710ca91a" (UID: "590a79d8-8970-45c0-9cd5-a5a5710ca91a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:01:59.461601 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:59.461568    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/590a79d8-8970-45c0-9cd5-a5a5710ca91a-kube-api-access-x4q6x" pod "590a79d8-8970-45c0-9cd5-a5a5710ca91a" (UID: "590a79d8-8970-45c0-9cd5-a5a5710ca91a"). InnerVolumeSpecName "kube-api-access-x4q6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:01:59.465579 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:59.465558    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/590a79d8-8970-45c0-9cd5-a5a5710ca91a-util" pod "590a79d8-8970-45c0-9cd5-a5a5710ca91a" (UID: "590a79d8-8970-45c0-9cd5-a5a5710ca91a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:01:59.560595 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:59.560570    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4q6x\" (UniqueName: \"kubernetes.io/projected/590a79d8-8970-45c0-9cd5-a5a5710ca91a-kube-api-access-x4q6x\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:59.560595 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:59.560596    4942 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/590a79d8-8970-45c0-9cd5-a5a5710ca91a-bundle\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:01:59.560735 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:01:59.560605    4942 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/590a79d8-8970-45c0-9cd5-a5a5710ca91a-util\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:02:00.183075 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:00.183035    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs" event={"ID":"590a79d8-8970-45c0-9cd5-a5a5710ca91a","Type":"ContainerDied","Data":"bdfcac3113daf87840fcb40c01018231e98e3920505d64f03182656231a4a3c2"}
Nov 02 12:02:00.183473 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:00.183084    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cgwdjs"
Nov 02 12:02:00.183473 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:00.183095    4942 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bdfcac3113daf87840fcb40c01018231e98e3920505d64f03182656231a4a3c2"
Nov 02 12:02:00.982002 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:02:00.981969    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:02:02.712006 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.711972    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"]
Nov 02 12:02:02.712373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712259    4942 cpu_manager.go:414] "containerMap: removing container" podUID="f95cad35-61f5-4419-9533-c2f9add0d73e" containerName="extract"
Nov 02 12:02:02.712373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712270    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f95cad35-61f5-4419-9533-c2f9add0d73e" containerName="extract"
Nov 02 12:02:02.712373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712280    4942 cpu_manager.go:414] "containerMap: removing container" podUID="a322fc87-6c9f-40a4-9d90-90a370bcdb01" containerName="util"
Nov 02 12:02:02.712373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712285    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a322fc87-6c9f-40a4-9d90-90a370bcdb01" containerName="util"
Nov 02 12:02:02.712373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712293    4942 cpu_manager.go:414] "containerMap: removing container" podUID="cca579e9-688a-4e73-8682-1a30b418ad54" containerName="pull"
Nov 02 12:02:02.712373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712313    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cca579e9-688a-4e73-8682-1a30b418ad54" containerName="pull"
Nov 02 12:02:02.712373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712323    4942 cpu_manager.go:414] "containerMap: removing container" podUID="8722ab06-1b04-4396-9e36-2c4ba23c9bc5" containerName="util"
Nov 02 12:02:02.712373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712328    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8722ab06-1b04-4396-9e36-2c4ba23c9bc5" containerName="util"
Nov 02 12:02:02.712373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712336    4942 cpu_manager.go:414] "containerMap: removing container" podUID="53c00485-1170-4977-919a-8d463b407845" containerName="util"
Nov 02 12:02:02.712373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712340    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53c00485-1170-4977-919a-8d463b407845" containerName="util"
Nov 02 12:02:02.712373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712346    4942 cpu_manager.go:414] "containerMap: removing container" podUID="cca579e9-688a-4e73-8682-1a30b418ad54" containerName="extract"
Nov 02 12:02:02.712373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712350    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cca579e9-688a-4e73-8682-1a30b418ad54" containerName="extract"
Nov 02 12:02:02.712373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712357    4942 cpu_manager.go:414] "containerMap: removing container" podUID="b556fe62-b13c-4c41-b452-9543d5dc01d7" containerName="util"
Nov 02 12:02:02.712373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712375    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b556fe62-b13c-4c41-b452-9543d5dc01d7" containerName="util"
Nov 02 12:02:02.712373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712381    4942 cpu_manager.go:414] "containerMap: removing container" podUID="590a79d8-8970-45c0-9cd5-a5a5710ca91a" containerName="extract"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712387    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="590a79d8-8970-45c0-9cd5-a5a5710ca91a" containerName="extract"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712396    4942 cpu_manager.go:414] "containerMap: removing container" podUID="b556fe62-b13c-4c41-b452-9543d5dc01d7" containerName="extract"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712400    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b556fe62-b13c-4c41-b452-9543d5dc01d7" containerName="extract"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712408    4942 cpu_manager.go:414] "containerMap: removing container" podUID="f95cad35-61f5-4419-9533-c2f9add0d73e" containerName="pull"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712412    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f95cad35-61f5-4419-9533-c2f9add0d73e" containerName="pull"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712418    4942 cpu_manager.go:414] "containerMap: removing container" podUID="8722ab06-1b04-4396-9e36-2c4ba23c9bc5" containerName="extract"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712422    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8722ab06-1b04-4396-9e36-2c4ba23c9bc5" containerName="extract"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712436    4942 cpu_manager.go:414] "containerMap: removing container" podUID="cca579e9-688a-4e73-8682-1a30b418ad54" containerName="util"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712440    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cca579e9-688a-4e73-8682-1a30b418ad54" containerName="util"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712446    4942 cpu_manager.go:414] "containerMap: removing container" podUID="8722ab06-1b04-4396-9e36-2c4ba23c9bc5" containerName="pull"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712450    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8722ab06-1b04-4396-9e36-2c4ba23c9bc5" containerName="pull"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712456    4942 cpu_manager.go:414] "containerMap: removing container" podUID="53c00485-1170-4977-919a-8d463b407845" containerName="extract"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712460    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53c00485-1170-4977-919a-8d463b407845" containerName="extract"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712467    4942 cpu_manager.go:414] "containerMap: removing container" podUID="f95cad35-61f5-4419-9533-c2f9add0d73e" containerName="util"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712471    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f95cad35-61f5-4419-9533-c2f9add0d73e" containerName="util"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712476    4942 cpu_manager.go:414] "containerMap: removing container" podUID="53c00485-1170-4977-919a-8d463b407845" containerName="pull"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712480    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53c00485-1170-4977-919a-8d463b407845" containerName="pull"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712485    4942 cpu_manager.go:414] "containerMap: removing container" podUID="a322fc87-6c9f-40a4-9d90-90a370bcdb01" containerName="extract"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712489    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a322fc87-6c9f-40a4-9d90-90a370bcdb01" containerName="extract"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712499    4942 cpu_manager.go:414] "containerMap: removing container" podUID="b556fe62-b13c-4c41-b452-9543d5dc01d7" containerName="pull"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712503    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b556fe62-b13c-4c41-b452-9543d5dc01d7" containerName="pull"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712508    4942 cpu_manager.go:414] "containerMap: removing container" podUID="590a79d8-8970-45c0-9cd5-a5a5710ca91a" containerName="pull"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712512    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="590a79d8-8970-45c0-9cd5-a5a5710ca91a" containerName="pull"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712542    4942 cpu_manager.go:414] "containerMap: removing container" podUID="a322fc87-6c9f-40a4-9d90-90a370bcdb01" containerName="pull"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712548    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a322fc87-6c9f-40a4-9d90-90a370bcdb01" containerName="pull"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712555    4942 cpu_manager.go:414] "containerMap: removing container" podUID="590a79d8-8970-45c0-9cd5-a5a5710ca91a" containerName="util"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712559    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="590a79d8-8970-45c0-9cd5-a5a5710ca91a" containerName="util"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712610    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="cca579e9-688a-4e73-8682-1a30b418ad54" containerName="extract"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712618    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="590a79d8-8970-45c0-9cd5-a5a5710ca91a" containerName="extract"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712624    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="b556fe62-b13c-4c41-b452-9543d5dc01d7" containerName="extract"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712630    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="8722ab06-1b04-4396-9e36-2c4ba23c9bc5" containerName="extract"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712637    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="53c00485-1170-4977-919a-8d463b407845" containerName="extract"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712644    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="a322fc87-6c9f-40a4-9d90-90a370bcdb01" containerName="extract"
Nov 02 12:02:02.712797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.712651    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="f95cad35-61f5-4419-9533-c2f9add0d73e" containerName="extract"
Nov 02 12:02:02.714519 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.714503    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:02.768197 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.768169    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 02 12:02:02.768442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.768417    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-qclbq\""
Nov 02 12:02:02.768498 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.768454    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 02 12:02:02.817704 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.817677    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 02 12:02:02.817910 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.817897    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 02 12:02:02.817956 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.817939    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 02 12:02:02.836182 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.836157    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"]
Nov 02 12:02:02.887228 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.887201    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/712a053c-95db-4eb2-bdbf-04ee490376af-metrics-cert\") pod \"opendatahub-operator-controller-manager-9776d789f-fm4pv\" (UID: \"712a053c-95db-4eb2-bdbf-04ee490376af\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:02.887361 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.887235    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/712a053c-95db-4eb2-bdbf-04ee490376af-webhook-cert\") pod \"opendatahub-operator-controller-manager-9776d789f-fm4pv\" (UID: \"712a053c-95db-4eb2-bdbf-04ee490376af\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:02.887361 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.887253    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/712a053c-95db-4eb2-bdbf-04ee490376af-manifests\") pod \"opendatahub-operator-controller-manager-9776d789f-fm4pv\" (UID: \"712a053c-95db-4eb2-bdbf-04ee490376af\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:02.887361 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.887334    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpb2k\" (UniqueName: \"kubernetes.io/projected/712a053c-95db-4eb2-bdbf-04ee490376af-kube-api-access-bpb2k\") pod \"opendatahub-operator-controller-manager-9776d789f-fm4pv\" (UID: \"712a053c-95db-4eb2-bdbf-04ee490376af\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:02.887494 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.887391    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/712a053c-95db-4eb2-bdbf-04ee490376af-tmp\") pod \"opendatahub-operator-controller-manager-9776d789f-fm4pv\" (UID: \"712a053c-95db-4eb2-bdbf-04ee490376af\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:02.887494 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.887425    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/712a053c-95db-4eb2-bdbf-04ee490376af-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9776d789f-fm4pv\" (UID: \"712a053c-95db-4eb2-bdbf-04ee490376af\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:02.988045 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.987972    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/712a053c-95db-4eb2-bdbf-04ee490376af-metrics-cert\") pod \"opendatahub-operator-controller-manager-9776d789f-fm4pv\" (UID: \"712a053c-95db-4eb2-bdbf-04ee490376af\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:02.988045 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.988009    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/712a053c-95db-4eb2-bdbf-04ee490376af-webhook-cert\") pod \"opendatahub-operator-controller-manager-9776d789f-fm4pv\" (UID: \"712a053c-95db-4eb2-bdbf-04ee490376af\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:02.988045 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.988026    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/712a053c-95db-4eb2-bdbf-04ee490376af-manifests\") pod \"opendatahub-operator-controller-manager-9776d789f-fm4pv\" (UID: \"712a053c-95db-4eb2-bdbf-04ee490376af\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:02.988322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.988055    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpb2k\" (UniqueName: \"kubernetes.io/projected/712a053c-95db-4eb2-bdbf-04ee490376af-kube-api-access-bpb2k\") pod \"opendatahub-operator-controller-manager-9776d789f-fm4pv\" (UID: \"712a053c-95db-4eb2-bdbf-04ee490376af\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:02.988322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.988097    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/712a053c-95db-4eb2-bdbf-04ee490376af-tmp\") pod \"opendatahub-operator-controller-manager-9776d789f-fm4pv\" (UID: \"712a053c-95db-4eb2-bdbf-04ee490376af\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:02.988322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.988138    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/712a053c-95db-4eb2-bdbf-04ee490376af-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9776d789f-fm4pv\" (UID: \"712a053c-95db-4eb2-bdbf-04ee490376af\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:02.988497 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.988417    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/712a053c-95db-4eb2-bdbf-04ee490376af-manifests\") pod \"opendatahub-operator-controller-manager-9776d789f-fm4pv\" (UID: \"712a053c-95db-4eb2-bdbf-04ee490376af\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:02.990594 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.990562    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/712a053c-95db-4eb2-bdbf-04ee490376af-tmp\") pod \"opendatahub-operator-controller-manager-9776d789f-fm4pv\" (UID: \"712a053c-95db-4eb2-bdbf-04ee490376af\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:02.990730 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.990629    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/712a053c-95db-4eb2-bdbf-04ee490376af-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9776d789f-fm4pv\" (UID: \"712a053c-95db-4eb2-bdbf-04ee490376af\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:02.990730 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.990722    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/712a053c-95db-4eb2-bdbf-04ee490376af-metrics-cert\") pod \"opendatahub-operator-controller-manager-9776d789f-fm4pv\" (UID: \"712a053c-95db-4eb2-bdbf-04ee490376af\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:02.990881 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:02.990817    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/712a053c-95db-4eb2-bdbf-04ee490376af-webhook-cert\") pod \"opendatahub-operator-controller-manager-9776d789f-fm4pv\" (UID: \"712a053c-95db-4eb2-bdbf-04ee490376af\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:03.053212 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:03.053184    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpb2k\" (UniqueName: \"kubernetes.io/projected/712a053c-95db-4eb2-bdbf-04ee490376af-kube-api-access-bpb2k\") pod \"opendatahub-operator-controller-manager-9776d789f-fm4pv\" (UID: \"712a053c-95db-4eb2-bdbf-04ee490376af\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:03.326948 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:03.326879    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:03.906783 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:03.906721    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"]
Nov 02 12:02:03.908999 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:02:03.908962    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod712a053c_95db_4eb2_bdbf_04ee490376af.slice/crio-02084c0144f7b7e82e3df7f3e9a749a6068065dd8efd27983d35d8fbcb41346e WatchSource:0}: Error finding container 02084c0144f7b7e82e3df7f3e9a749a6068065dd8efd27983d35d8fbcb41346e: Status 404 returned error can't find the container with id 02084c0144f7b7e82e3df7f3e9a749a6068065dd8efd27983d35d8fbcb41346e
Nov 02 12:02:04.208207 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:04.208130    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv" event={"ID":"712a053c-95db-4eb2-bdbf-04ee490376af","Type":"ContainerStarted","Data":"02084c0144f7b7e82e3df7f3e9a749a6068065dd8efd27983d35d8fbcb41346e"}
Nov 02 12:02:07.226219 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:07.226190    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv" event={"ID":"712a053c-95db-4eb2-bdbf-04ee490376af","Type":"ContainerStarted","Data":"62d3130c9532678f7906208783e0f886a129647c02c4b0755c415bb7f257a507"}
Nov 02 12:02:08.231794 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:08.231760    4942 generic.go:356] "Generic (PLEG): container finished" podID="712a053c-95db-4eb2-bdbf-04ee490376af" containerID="62d3130c9532678f7906208783e0f886a129647c02c4b0755c415bb7f257a507" exitCode=0
Nov 02 12:02:08.232172 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:08.231806    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv" event={"ID":"712a053c-95db-4eb2-bdbf-04ee490376af","Type":"ContainerDied","Data":"62d3130c9532678f7906208783e0f886a129647c02c4b0755c415bb7f257a507"}
Nov 02 12:02:09.238221 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:09.238178    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv" event={"ID":"712a053c-95db-4eb2-bdbf-04ee490376af","Type":"ContainerStarted","Data":"a9a27325efb47b8be6758145b1fb48c89833cde1b4502f510d7e1a6792ef5b19"}
Nov 02 12:02:09.238621 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:09.238387    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:09.474363 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:09.474238    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv" podStartSLOduration=2.905936698 podStartE2EDuration="7.47422414s" totalImagesPullingTime="4.095504569s" totalInitContainerRuntime="472.782873ms" isStatefulPod=false podCreationTimestamp="2026-11-02 12:02:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:02:09.424196893 +0000 UTC m=+355.995546650" watchObservedRunningTime="2026-11-02 12:02:09.47422414 +0000 UTC m=+356.045573896"
Nov 02 12:02:12.636089 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:12.636058    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-vktcs"]
Nov 02 12:02:12.642183 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:12.642166    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vktcs"
Nov 02 12:02:12.672469 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:12.672445    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-678kb\" (UniqueName: \"kubernetes.io/projected/da858d4c-2fba-41ef-a18a-2ee3d0dcbbea-kube-api-access-678kb\") pod \"limitador-operator-controller-manager-5778bb4449-vktcs\" (UID: \"da858d4c-2fba-41ef-a18a-2ee3d0dcbbea\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vktcs"
Nov 02 12:02:12.746801 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:12.746768    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 02 12:02:12.746921 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:12.746858    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 02 12:02:12.746921 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:12.746877    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-zxrgb\""
Nov 02 12:02:12.748669 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:12.748653    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-vktcs"]
Nov 02 12:02:12.773453 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:12.773433    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-678kb\" (UniqueName: \"kubernetes.io/projected/da858d4c-2fba-41ef-a18a-2ee3d0dcbbea-kube-api-access-678kb\") pod \"limitador-operator-controller-manager-5778bb4449-vktcs\" (UID: \"da858d4c-2fba-41ef-a18a-2ee3d0dcbbea\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vktcs"
Nov 02 12:02:12.834366 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:12.834342    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-678kb\" (UniqueName: \"kubernetes.io/projected/da858d4c-2fba-41ef-a18a-2ee3d0dcbbea-kube-api-access-678kb\") pod \"limitador-operator-controller-manager-5778bb4449-vktcs\" (UID: \"da858d4c-2fba-41ef-a18a-2ee3d0dcbbea\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vktcs"
Nov 02 12:02:12.954517 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:12.954456    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vktcs"
Nov 02 12:02:13.326404 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:13.326328    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-vktcs"]
Nov 02 12:02:13.356405 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:02:13.356378    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda858d4c_2fba_41ef_a18a_2ee3d0dcbbea.slice/crio-8c4272ad493c3f817e0e5173215229c3ba1fff1fe490a98ce981564b3f7986fc WatchSource:0}: Error finding container 8c4272ad493c3f817e0e5173215229c3ba1fff1fe490a98ce981564b3f7986fc: Status 404 returned error can't find the container with id 8c4272ad493c3f817e0e5173215229c3ba1fff1fe490a98ce981564b3f7986fc
Nov 02 12:02:14.261096 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:14.261052    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vktcs" event={"ID":"da858d4c-2fba-41ef-a18a-2ee3d0dcbbea","Type":"ContainerStarted","Data":"8c4272ad493c3f817e0e5173215229c3ba1fff1fe490a98ce981564b3f7986fc"}
Nov 02 12:02:17.281368 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:17.281259    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vktcs" event={"ID":"da858d4c-2fba-41ef-a18a-2ee3d0dcbbea","Type":"ContainerStarted","Data":"d0186b673502cd327411b3203644989c644f0ebb1bdaad18515502df003764ea"}
Nov 02 12:02:17.281368 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:17.281352    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vktcs"
Nov 02 12:02:17.301136 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:17.301106    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-56ffx"]
Nov 02 12:02:17.304817 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:17.304798    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-56ffx"
Nov 02 12:02:17.399243 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:17.399212    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Nov 02 12:02:17.399476 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:17.399460    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-tqbns\""
Nov 02 12:02:17.401848 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:17.401819    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-56ffx"]
Nov 02 12:02:17.410911 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:17.410884    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wqv4\" (UniqueName: \"kubernetes.io/projected/58467a64-e86d-42d0-9ef7-8fcafca6a9e5-kube-api-access-5wqv4\") pod \"dns-operator-controller-manager-8557cbdb84-56ffx\" (UID: \"58467a64-e86d-42d0-9ef7-8fcafca6a9e5\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-56ffx"
Nov 02 12:02:17.427234 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:17.427192    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vktcs" podStartSLOduration=1.798546564 podStartE2EDuration="5.427179621s" totalImagesPullingTime="3.628633057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:02:17.425715626 +0000 UTC m=+363.997065386" watchObservedRunningTime="2026-11-02 12:02:17.427179621 +0000 UTC m=+363.998529378"
Nov 02 12:02:17.511557 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:17.511530    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wqv4\" (UniqueName: \"kubernetes.io/projected/58467a64-e86d-42d0-9ef7-8fcafca6a9e5-kube-api-access-5wqv4\") pod \"dns-operator-controller-manager-8557cbdb84-56ffx\" (UID: \"58467a64-e86d-42d0-9ef7-8fcafca6a9e5\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-56ffx"
Nov 02 12:02:17.585390 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:17.585315    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wqv4\" (UniqueName: \"kubernetes.io/projected/58467a64-e86d-42d0-9ef7-8fcafca6a9e5-kube-api-access-5wqv4\") pod \"dns-operator-controller-manager-8557cbdb84-56ffx\" (UID: \"58467a64-e86d-42d0-9ef7-8fcafca6a9e5\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-56ffx"
Nov 02 12:02:17.617353 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:17.617320    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-56ffx"
Nov 02 12:02:18.044239 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:18.044206    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-56ffx"]
Nov 02 12:02:18.045790 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:02:18.045764    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58467a64_e86d_42d0_9ef7_8fcafca6a9e5.slice/crio-be9279cbc9d47a18613eb5338b5a6639c4f00af1bdc71f0319f3404d90d1e608 WatchSource:0}: Error finding container be9279cbc9d47a18613eb5338b5a6639c4f00af1bdc71f0319f3404d90d1e608: Status 404 returned error can't find the container with id be9279cbc9d47a18613eb5338b5a6639c4f00af1bdc71f0319f3404d90d1e608
Nov 02 12:02:18.287229 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:18.287199    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-56ffx" event={"ID":"58467a64-e86d-42d0-9ef7-8fcafca6a9e5","Type":"ContainerStarted","Data":"be9279cbc9d47a18613eb5338b5a6639c4f00af1bdc71f0319f3404d90d1e608"}
Nov 02 12:02:20.245575 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:20.245545    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-9776d789f-fm4pv"
Nov 02 12:02:20.801453 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:20.801412    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-49lgm"]
Nov 02 12:02:20.805026 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:20.805001    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-49lgm"
Nov 02 12:02:20.842053 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:20.842022    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75479a23-6231-4bcc-bfdb-984a36f3616d-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-49lgm\" (UID: \"75479a23-6231-4bcc-bfdb-984a36f3616d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-49lgm"
Nov 02 12:02:20.842170 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:20.842131    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsdjw\" (UniqueName: \"kubernetes.io/projected/75479a23-6231-4bcc-bfdb-984a36f3616d-kube-api-access-xsdjw\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-49lgm\" (UID: \"75479a23-6231-4bcc-bfdb-984a36f3616d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-49lgm"
Nov 02 12:02:20.864722 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:20.864700    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-8ch7m\""
Nov 02 12:02:20.864722 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:20.864706    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Nov 02 12:02:20.864998 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:20.864982    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Nov 02 12:02:20.872009 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:20.871986    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-49lgm"]
Nov 02 12:02:20.943484 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:20.943456    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsdjw\" (UniqueName: \"kubernetes.io/projected/75479a23-6231-4bcc-bfdb-984a36f3616d-kube-api-access-xsdjw\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-49lgm\" (UID: \"75479a23-6231-4bcc-bfdb-984a36f3616d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-49lgm"
Nov 02 12:02:20.943629 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:20.943516    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75479a23-6231-4bcc-bfdb-984a36f3616d-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-49lgm\" (UID: \"75479a23-6231-4bcc-bfdb-984a36f3616d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-49lgm"
Nov 02 12:02:20.943870 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:20.943853    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75479a23-6231-4bcc-bfdb-984a36f3616d-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-49lgm\" (UID: \"75479a23-6231-4bcc-bfdb-984a36f3616d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-49lgm"
Nov 02 12:02:21.007666 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:21.007633    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsdjw\" (UniqueName: \"kubernetes.io/projected/75479a23-6231-4bcc-bfdb-984a36f3616d-kube-api-access-xsdjw\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-49lgm\" (UID: \"75479a23-6231-4bcc-bfdb-984a36f3616d\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-49lgm"
Nov 02 12:02:21.116404 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:21.116372    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-49lgm"
Nov 02 12:02:21.303859 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:21.303823    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-56ffx" event={"ID":"58467a64-e86d-42d0-9ef7-8fcafca6a9e5","Type":"ContainerStarted","Data":"c3c7df2185eb806904191427d701f6a5c26dd797ac278b5bdc09ff3fb0a77639"}
Nov 02 12:02:21.304252 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:21.303893    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-56ffx"
Nov 02 12:02:21.426703 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:21.426610    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-56ffx" podStartSLOduration=1.913436791 podStartE2EDuration="4.426594564s" totalImagesPullingTime="2.513157773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:02:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:02:21.421632646 +0000 UTC m=+367.992982403" watchObservedRunningTime="2026-11-02 12:02:21.426594564 +0000 UTC m=+367.997944320"
Nov 02 12:02:21.549664 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:21.549623    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-49lgm"]
Nov 02 12:02:21.552206 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:02:21.552174    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75479a23_6231_4bcc_bfdb_984a36f3616d.slice/crio-11ceee2e95d9e194486c170f06c4f49341c967e802a2f94a238d30da1f5519df WatchSource:0}: Error finding container 11ceee2e95d9e194486c170f06c4f49341c967e802a2f94a238d30da1f5519df: Status 404 returned error can't find the container with id 11ceee2e95d9e194486c170f06c4f49341c967e802a2f94a238d30da1f5519df
Nov 02 12:02:22.309339 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:22.309255    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-49lgm" event={"ID":"75479a23-6231-4bcc-bfdb-984a36f3616d","Type":"ContainerStarted","Data":"11ceee2e95d9e194486c170f06c4f49341c967e802a2f94a238d30da1f5519df"}
Nov 02 12:02:24.324180 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:24.324140    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-49lgm" event={"ID":"75479a23-6231-4bcc-bfdb-984a36f3616d","Type":"ContainerStarted","Data":"d122c3013fb2b04c2e1b64ff274258f8745967cdb2f5c8e86258ec8a5d8de1d1"}
Nov 02 12:02:24.542402 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:24.542274    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-49lgm" podStartSLOduration=1.9714857399999999 podStartE2EDuration="4.542262386s" totalImagesPullingTime="2.570776646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:02:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:02:24.542009682 +0000 UTC m=+371.113359438" watchObservedRunningTime="2026-11-02 12:02:24.542262386 +0000 UTC m=+371.113612143"
Nov 02 12:02:25.230353 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:25.230317    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc"]
Nov 02 12:02:25.233554 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:25.233532    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc"
Nov 02 12:02:25.283497 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:25.283461    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6555j\" (UniqueName: \"kubernetes.io/projected/fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902-kube-api-access-6555j\") pod \"kuadrant-operator-controller-manager-678d87766f-7wrsc\" (UID: \"fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc"
Nov 02 12:02:25.283654 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:25.283520    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-7wrsc\" (UID: \"fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc"
Nov 02 12:02:25.288242 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:25.288219    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gzmrw\""
Nov 02 12:02:25.340397 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:25.340366    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc"]
Nov 02 12:02:25.385018 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:25.384975    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6555j\" (UniqueName: \"kubernetes.io/projected/fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902-kube-api-access-6555j\") pod \"kuadrant-operator-controller-manager-678d87766f-7wrsc\" (UID: \"fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc"
Nov 02 12:02:25.385207 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:25.385032    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-7wrsc\" (UID: \"fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc"
Nov 02 12:02:25.385482 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:25.385462    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-7wrsc\" (UID: \"fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc"
Nov 02 12:02:25.482033 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:25.481957    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6555j\" (UniqueName: \"kubernetes.io/projected/fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902-kube-api-access-6555j\") pod \"kuadrant-operator-controller-manager-678d87766f-7wrsc\" (UID: \"fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc"
Nov 02 12:02:25.545784 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:25.545752    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc"
Nov 02 12:02:25.976467 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:25.976418    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc"]
Nov 02 12:02:25.978804 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:02:25.978774    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf7d774_bb8f_4ee4_8a74_4fb7e7ebd902.slice/crio-ae7f3f0945b0b863a0c61e83214204d5ce8c43330d2f08e683b856226ed600d0 WatchSource:0}: Error finding container ae7f3f0945b0b863a0c61e83214204d5ce8c43330d2f08e683b856226ed600d0: Status 404 returned error can't find the container with id ae7f3f0945b0b863a0c61e83214204d5ce8c43330d2f08e683b856226ed600d0
Nov 02 12:02:26.335379 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:26.335276    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc" event={"ID":"fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902","Type":"ContainerStarted","Data":"ae7f3f0945b0b863a0c61e83214204d5ce8c43330d2f08e683b856226ed600d0"}
Nov 02 12:02:28.289678 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:28.289645    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-vktcs"
Nov 02 12:02:30.984450 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:02:30.984368    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:02:31.361951 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:31.361913    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc" event={"ID":"fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902","Type":"ContainerStarted","Data":"3a96408d7e9891ddd779fe7e0b9a6dd9b637b10e7c2c1e57703494720d3e08bf"}
Nov 02 12:02:31.362115 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:31.361987    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc"
Nov 02 12:02:31.581475 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:31.581420    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc" podStartSLOduration=1.949262628 podStartE2EDuration="6.581406406s" totalImagesPullingTime="4.632143778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:02:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:02:31.530648253 +0000 UTC m=+378.101998011" watchObservedRunningTime="2026-11-02 12:02:31.581406406 +0000 UTC m=+378.152756163"
Nov 02 12:02:32.311635 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:32.311603    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-56ffx"
Nov 02 12:02:41.062744 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:41.062695    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"]
Nov 02 12:02:41.068007 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:41.067984    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"
Nov 02 12:02:41.121837 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:41.121786    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 02 12:02:41.171673 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:41.171649    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-98q7s\""
Nov 02 12:02:41.173508 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:41.173485    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"]
Nov 02 12:02:41.212652 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:41.212627    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c9b2ae98-816e-453f-b7cd-b4122d871910-config-file\") pod \"limitador-limitador-74dfcfb894-zw4g2\" (UID: \"c9b2ae98-816e-453f-b7cd-b4122d871910\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"
Nov 02 12:02:41.212770 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:41.212672    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsfbn\" (UniqueName: \"kubernetes.io/projected/c9b2ae98-816e-453f-b7cd-b4122d871910-kube-api-access-jsfbn\") pod \"limitador-limitador-74dfcfb894-zw4g2\" (UID: \"c9b2ae98-816e-453f-b7cd-b4122d871910\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"
Nov 02 12:02:41.308768 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:41.308740    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"]
Nov 02 12:02:41.313290 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:41.313228    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c9b2ae98-816e-453f-b7cd-b4122d871910-config-file\") pod \"limitador-limitador-74dfcfb894-zw4g2\" (UID: \"c9b2ae98-816e-453f-b7cd-b4122d871910\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"
Nov 02 12:02:41.313290 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:41.313268    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsfbn\" (UniqueName: \"kubernetes.io/projected/c9b2ae98-816e-453f-b7cd-b4122d871910-kube-api-access-jsfbn\") pod \"limitador-limitador-74dfcfb894-zw4g2\" (UID: \"c9b2ae98-816e-453f-b7cd-b4122d871910\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"
Nov 02 12:02:41.313986 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:41.313968    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c9b2ae98-816e-453f-b7cd-b4122d871910-config-file\") pod \"limitador-limitador-74dfcfb894-zw4g2\" (UID: \"c9b2ae98-816e-453f-b7cd-b4122d871910\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"
Nov 02 12:02:41.377275 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:41.377248    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsfbn\" (UniqueName: \"kubernetes.io/projected/c9b2ae98-816e-453f-b7cd-b4122d871910-kube-api-access-jsfbn\") pod \"limitador-limitador-74dfcfb894-zw4g2\" (UID: \"c9b2ae98-816e-453f-b7cd-b4122d871910\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"
Nov 02 12:02:41.379025 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:41.379002    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"
Nov 02 12:02:41.780637 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:41.780609    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"]
Nov 02 12:02:41.782283 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:02:41.782250    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9b2ae98_816e_453f_b7cd_b4122d871910.slice/crio-03ca90cb6662e7f31180e762170d0ee14dcb5dd581ff5487ce20668ff776588c WatchSource:0}: Error finding container 03ca90cb6662e7f31180e762170d0ee14dcb5dd581ff5487ce20668ff776588c: Status 404 returned error can't find the container with id 03ca90cb6662e7f31180e762170d0ee14dcb5dd581ff5487ce20668ff776588c
Nov 02 12:02:42.194437 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.194405    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"]
Nov 02 12:02:42.197140 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.197122    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.250489 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.250462    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-s88cj\""
Nov 02 12:02:42.263347 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.263316    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"]
Nov 02 12:02:42.322686 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.322653    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.322686 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.322689    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.323249 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.322714    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.323249 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.322822    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.323249 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.322842    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.323249 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.322867    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.323249 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.322921    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.323249 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.322995    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.323249 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.323032    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhshq\" (UniqueName: \"kubernetes.io/projected/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-kube-api-access-hhshq\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.368340 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.368311    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc"
Nov 02 12:02:42.415066 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.415028    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-zw4g2" event={"ID":"c9b2ae98-816e-453f-b7cd-b4122d871910","Type":"ContainerStarted","Data":"03ca90cb6662e7f31180e762170d0ee14dcb5dd581ff5487ce20668ff776588c"}
Nov 02 12:02:42.423673 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.423644    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.423797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.423679    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.423797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.423714    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.423911 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.423889    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.423965 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.423944    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.424001 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.423977    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.424046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.424022    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.424099 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.424061    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.424213 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.424113    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhshq\" (UniqueName: \"kubernetes.io/projected/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-kube-api-access-hhshq\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.424275 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.424234    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.424474 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.424443    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.424599 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.424510    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.424599 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.424542    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.424599 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.424538    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.426093 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.426064    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.426369 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.426350    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.484413 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.484343    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.484532 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.484442    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhshq\" (UniqueName: \"kubernetes.io/projected/adac8dc5-74ea-423a-9ff3-ab12fe7c6585-kube-api-access-hhshq\") pod \"data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z\" (UID: \"adac8dc5-74ea-423a-9ff3-ab12fe7c6585\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.509338 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.509270    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:42.671173 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.671136    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-mhmp4"]
Nov 02 12:02:42.674394 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.674374    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-mhmp4"
Nov 02 12:02:42.729579 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.729549    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xwv5h\""
Nov 02 12:02:42.731290 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.731264    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-mhmp4"]
Nov 02 12:02:42.828282 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.828210    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxd6t\" (UniqueName: \"kubernetes.io/projected/39437d8e-b528-46d5-ba95-a18ab968214b-kube-api-access-hxd6t\") pod \"authorino-57f7f8659f-mhmp4\" (UID: \"39437d8e-b528-46d5-ba95-a18ab968214b\") " pod="kuadrant-system/authorino-57f7f8659f-mhmp4"
Nov 02 12:02:42.929786 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.929754    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxd6t\" (UniqueName: \"kubernetes.io/projected/39437d8e-b528-46d5-ba95-a18ab968214b-kube-api-access-hxd6t\") pod \"authorino-57f7f8659f-mhmp4\" (UID: \"39437d8e-b528-46d5-ba95-a18ab968214b\") " pod="kuadrant-system/authorino-57f7f8659f-mhmp4"
Nov 02 12:02:42.959076 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.959050    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"]
Nov 02 12:02:42.960727 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:02:42.960700    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadac8dc5_74ea_423a_9ff3_ab12fe7c6585.slice/crio-4cacaaa32b9bdb622ed30dc9641d2474d34984d7d8108934b67b4ad11968a92b WatchSource:0}: Error finding container 4cacaaa32b9bdb622ed30dc9641d2474d34984d7d8108934b67b4ad11968a92b: Status 404 returned error can't find the container with id 4cacaaa32b9bdb622ed30dc9641d2474d34984d7d8108934b67b4ad11968a92b
Nov 02 12:02:42.962903 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.962873    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:02:42.962989 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.962932    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:02:42.962989 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.962961    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:02:42.992764 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.992741    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxd6t\" (UniqueName: \"kubernetes.io/projected/39437d8e-b528-46d5-ba95-a18ab968214b-kube-api-access-hxd6t\") pod \"authorino-57f7f8659f-mhmp4\" (UID: \"39437d8e-b528-46d5-ba95-a18ab968214b\") " pod="kuadrant-system/authorino-57f7f8659f-mhmp4"
Nov 02 12:02:42.999402 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:42.999383    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-mhmp4"
Nov 02 12:02:43.380177 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:43.380153    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-mhmp4"]
Nov 02 12:02:43.381962 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:02:43.381931    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39437d8e_b528_46d5_ba95_a18ab968214b.slice/crio-9a4198a67d62279a2aeb2d4ca5f30588bf80f9d70ee6537d293d002eb506ada3 WatchSource:0}: Error finding container 9a4198a67d62279a2aeb2d4ca5f30588bf80f9d70ee6537d293d002eb506ada3: Status 404 returned error can't find the container with id 9a4198a67d62279a2aeb2d4ca5f30588bf80f9d70ee6537d293d002eb506ada3
Nov 02 12:02:43.421054 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:43.421023    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z" event={"ID":"adac8dc5-74ea-423a-9ff3-ab12fe7c6585","Type":"ContainerStarted","Data":"b2327ece41a275e56f2043c5fc6443bba86d4f87ec9bd6f3ef06bd7622fa0f5c"}
Nov 02 12:02:43.421191 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:43.421120    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z" event={"ID":"adac8dc5-74ea-423a-9ff3-ab12fe7c6585","Type":"ContainerStarted","Data":"4cacaaa32b9bdb622ed30dc9641d2474d34984d7d8108934b67b4ad11968a92b"}
Nov 02 12:02:43.422217 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:43.422194    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-mhmp4" event={"ID":"39437d8e-b528-46d5-ba95-a18ab968214b","Type":"ContainerStarted","Data":"9a4198a67d62279a2aeb2d4ca5f30588bf80f9d70ee6537d293d002eb506ada3"}
Nov 02 12:02:43.509917 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:43.509892    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:43.587753 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:43.587699    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z" podStartSLOduration=1.587683406 podStartE2EDuration="1.587683406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:02:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:02:43.587030036 +0000 UTC m=+390.158379793" watchObservedRunningTime="2026-11-02 12:02:43.587683406 +0000 UTC m=+390.159033163"
Nov 02 12:02:44.514550 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:44.514527    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:45.433641 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:45.433606    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-zw4g2" event={"ID":"c9b2ae98-816e-453f-b7cd-b4122d871910","Type":"ContainerStarted","Data":"4f66532cd52b9a294b09c4595cb30779b719782752ca2458576a2082b47f39fa"}
Nov 02 12:02:45.433981 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:45.433957    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:45.434859 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:45.434840    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z"
Nov 02 12:02:45.510655 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:45.510630    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-vn25l"]
Nov 02 12:02:45.512933 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:45.512913    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-vn25l"
Nov 02 12:02:45.550745 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:45.550704    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-zw4g2" podStartSLOduration=1.8834467510000001 podStartE2EDuration="4.550691501s" totalImagesPullingTime="2.66724475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:02:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:02:45.549333434 +0000 UTC m=+392.120683192" watchObservedRunningTime="2026-11-02 12:02:45.550691501 +0000 UTC m=+392.122041294"
Nov 02 12:02:45.569942 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:45.569921    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-vn25l"]
Nov 02 12:02:45.653025 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:45.652996    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4wxh\" (UniqueName: \"kubernetes.io/projected/d2a2667a-f64b-431a-a6d8-b60a67c4df50-kube-api-access-d4wxh\") pod \"authorino-56c6c9ddd5-vn25l\" (UID: \"d2a2667a-f64b-431a-a6d8-b60a67c4df50\") " pod="kuadrant-system/authorino-56c6c9ddd5-vn25l"
Nov 02 12:02:45.754061 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:45.753986    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4wxh\" (UniqueName: \"kubernetes.io/projected/d2a2667a-f64b-431a-a6d8-b60a67c4df50-kube-api-access-d4wxh\") pod \"authorino-56c6c9ddd5-vn25l\" (UID: \"d2a2667a-f64b-431a-a6d8-b60a67c4df50\") " pod="kuadrant-system/authorino-56c6c9ddd5-vn25l"
Nov 02 12:02:45.815514 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:45.815483    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4wxh\" (UniqueName: \"kubernetes.io/projected/d2a2667a-f64b-431a-a6d8-b60a67c4df50-kube-api-access-d4wxh\") pod \"authorino-56c6c9ddd5-vn25l\" (UID: \"d2a2667a-f64b-431a-a6d8-b60a67c4df50\") " pod="kuadrant-system/authorino-56c6c9ddd5-vn25l"
Nov 02 12:02:45.823147 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:45.823124    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-vn25l"
Nov 02 12:02:46.228690 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:46.228659    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-vn25l"]
Nov 02 12:02:46.439633 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:46.439592    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-vn25l" event={"ID":"d2a2667a-f64b-431a-a6d8-b60a67c4df50","Type":"ContainerStarted","Data":"32a361bea633ff129942a9be93d3922f5e23c9987e6b032b538b53dcf3d27a54"}
Nov 02 12:02:46.439787 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:46.439769    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"
Nov 02 12:02:48.453326 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:48.453252    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-mhmp4" event={"ID":"39437d8e-b528-46d5-ba95-a18ab968214b","Type":"ContainerStarted","Data":"54f6fd1248f3647145bd98fec14b56dd0fff97fcaabf882bea3bd5eb34ae37e7"}
Nov 02 12:02:48.575651 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:48.575598    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-mhmp4" podStartSLOduration=1.7322655409999999 podStartE2EDuration="6.575578625s" totalImagesPullingTime="4.843313084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:02:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:02:48.574897759 +0000 UTC m=+395.146247519" watchObservedRunningTime="2026-11-02 12:02:48.575578625 +0000 UTC m=+395.146928384"
Nov 02 12:02:49.460761 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:49.460723    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-vn25l" event={"ID":"d2a2667a-f64b-431a-a6d8-b60a67c4df50","Type":"ContainerStarted","Data":"77c6a7d38e5ebb7e30b6b8a7371d2c853ea08107168e6dc46c433c01d77ff99e"}
Nov 02 12:02:49.588250 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:49.588199    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-vn25l" podStartSLOduration=2.267940692 podStartE2EDuration="4.588185391s" totalImagesPullingTime="2.320244699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:02:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:02:49.585456201 +0000 UTC m=+396.156805958" watchObservedRunningTime="2026-11-02 12:02:49.588185391 +0000 UTC m=+396.159535147"
Nov 02 12:02:49.637997 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:49.637965    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-mhmp4"]
Nov 02 12:02:50.465930 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:50.465892    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-mhmp4" podUID="39437d8e-b528-46d5-ba95-a18ab968214b" containerName="authorino" containerID="cri-o://54f6fd1248f3647145bd98fec14b56dd0fff97fcaabf882bea3bd5eb34ae37e7" gracePeriod=30
Nov 02 12:02:50.716740 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:50.716683    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-mhmp4"
Nov 02 12:02:50.802662 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:50.802634    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/39437d8e-b528-46d5-ba95-a18ab968214b-kube-api-access-hxd6t\" (UniqueName: \"kubernetes.io/projected/39437d8e-b528-46d5-ba95-a18ab968214b-kube-api-access-hxd6t\") pod \"39437d8e-b528-46d5-ba95-a18ab968214b\" (UID: \"39437d8e-b528-46d5-ba95-a18ab968214b\") "
Nov 02 12:02:50.804698 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:50.804677    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39437d8e-b528-46d5-ba95-a18ab968214b-kube-api-access-hxd6t" pod "39437d8e-b528-46d5-ba95-a18ab968214b" (UID: "39437d8e-b528-46d5-ba95-a18ab968214b"). InnerVolumeSpecName "kube-api-access-hxd6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:02:50.904246 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:50.904218    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxd6t\" (UniqueName: \"kubernetes.io/projected/39437d8e-b528-46d5-ba95-a18ab968214b-kube-api-access-hxd6t\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:02:51.472027 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:51.471992    4942 generic.go:356] "Generic (PLEG): container finished" podID="39437d8e-b528-46d5-ba95-a18ab968214b" containerID="54f6fd1248f3647145bd98fec14b56dd0fff97fcaabf882bea3bd5eb34ae37e7" exitCode=0
Nov 02 12:02:51.472436 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:51.472034    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-mhmp4"
Nov 02 12:02:51.472436 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:51.472048    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-mhmp4" event={"ID":"39437d8e-b528-46d5-ba95-a18ab968214b","Type":"ContainerDied","Data":"54f6fd1248f3647145bd98fec14b56dd0fff97fcaabf882bea3bd5eb34ae37e7"}
Nov 02 12:02:51.472436 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:51.472087    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-mhmp4" event={"ID":"39437d8e-b528-46d5-ba95-a18ab968214b","Type":"ContainerDied","Data":"9a4198a67d62279a2aeb2d4ca5f30588bf80f9d70ee6537d293d002eb506ada3"}
Nov 02 12:02:51.472436 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:51.472107    4942 scope.go:122] "RemoveContainer" containerID="54f6fd1248f3647145bd98fec14b56dd0fff97fcaabf882bea3bd5eb34ae37e7"
Nov 02 12:02:51.481408 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:51.481292    4942 scope.go:122] "RemoveContainer" containerID="54f6fd1248f3647145bd98fec14b56dd0fff97fcaabf882bea3bd5eb34ae37e7"
Nov 02 12:02:51.481677 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:02:51.481652    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54f6fd1248f3647145bd98fec14b56dd0fff97fcaabf882bea3bd5eb34ae37e7\": container with ID starting with 54f6fd1248f3647145bd98fec14b56dd0fff97fcaabf882bea3bd5eb34ae37e7 not found: ID does not exist" containerID="54f6fd1248f3647145bd98fec14b56dd0fff97fcaabf882bea3bd5eb34ae37e7"
Nov 02 12:02:51.481739 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:51.481679    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54f6fd1248f3647145bd98fec14b56dd0fff97fcaabf882bea3bd5eb34ae37e7"} err="failed to get container status \"54f6fd1248f3647145bd98fec14b56dd0fff97fcaabf882bea3bd5eb34ae37e7\": rpc error: code = NotFound desc = could not find container \"54f6fd1248f3647145bd98fec14b56dd0fff97fcaabf882bea3bd5eb34ae37e7\": container with ID starting with 54f6fd1248f3647145bd98fec14b56dd0fff97fcaabf882bea3bd5eb34ae37e7 not found: ID does not exist"
Nov 02 12:02:51.634373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:51.634342    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-mhmp4"]
Nov 02 12:02:51.640981 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:51.640955    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-mhmp4"]
Nov 02 12:02:53.541322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:53.541261    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="39437d8e-b528-46d5-ba95-a18ab968214b" path="/var/lib/kubelet/pods/39437d8e-b528-46d5-ba95-a18ab968214b/volumes"
Nov 02 12:02:57.447143 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:02:57.447114    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"
Nov 02 12:03:00.981981 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:03:00.981941    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:03:01.301622 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:01.301548    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"]
Nov 02 12:03:01.301794 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:01.301755    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-zw4g2" podUID="c9b2ae98-816e-453f-b7cd-b4122d871910" containerName="limitador" containerID="cri-o://4f66532cd52b9a294b09c4595cb30779b719782752ca2458576a2082b47f39fa" gracePeriod=30
Nov 02 12:03:02.268119 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:02.268094    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"
Nov 02 12:03:02.293924 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:02.293902    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c9b2ae98-816e-453f-b7cd-b4122d871910-config-file\" (UniqueName: \"kubernetes.io/configmap/c9b2ae98-816e-453f-b7cd-b4122d871910-config-file\") pod \"c9b2ae98-816e-453f-b7cd-b4122d871910\" (UID: \"c9b2ae98-816e-453f-b7cd-b4122d871910\") "
Nov 02 12:03:02.294036 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:02.293966    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9b2ae98-816e-453f-b7cd-b4122d871910-kube-api-access-jsfbn\" (UniqueName: \"kubernetes.io/projected/c9b2ae98-816e-453f-b7cd-b4122d871910-kube-api-access-jsfbn\") pod \"c9b2ae98-816e-453f-b7cd-b4122d871910\" (UID: \"c9b2ae98-816e-453f-b7cd-b4122d871910\") "
Nov 02 12:03:02.294275 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:02.294252    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9b2ae98-816e-453f-b7cd-b4122d871910-config-file" pod "c9b2ae98-816e-453f-b7cd-b4122d871910" (UID: "c9b2ae98-816e-453f-b7cd-b4122d871910"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:03:02.295870 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:02.295846    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9b2ae98-816e-453f-b7cd-b4122d871910-kube-api-access-jsfbn" pod "c9b2ae98-816e-453f-b7cd-b4122d871910" (UID: "c9b2ae98-816e-453f-b7cd-b4122d871910"). InnerVolumeSpecName "kube-api-access-jsfbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:03:02.395106 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:02.395051    4942 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c9b2ae98-816e-453f-b7cd-b4122d871910-config-file\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:03:02.395106 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:02.395076    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsfbn\" (UniqueName: \"kubernetes.io/projected/c9b2ae98-816e-453f-b7cd-b4122d871910-kube-api-access-jsfbn\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:03:02.527163 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:02.527127    4942 generic.go:356] "Generic (PLEG): container finished" podID="c9b2ae98-816e-453f-b7cd-b4122d871910" containerID="4f66532cd52b9a294b09c4595cb30779b719782752ca2458576a2082b47f39fa" exitCode=0
Nov 02 12:03:02.527355 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:02.527179    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"
Nov 02 12:03:02.527355 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:02.527206    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-zw4g2" event={"ID":"c9b2ae98-816e-453f-b7cd-b4122d871910","Type":"ContainerDied","Data":"4f66532cd52b9a294b09c4595cb30779b719782752ca2458576a2082b47f39fa"}
Nov 02 12:03:02.527355 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:02.527264    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-zw4g2" event={"ID":"c9b2ae98-816e-453f-b7cd-b4122d871910","Type":"ContainerDied","Data":"03ca90cb6662e7f31180e762170d0ee14dcb5dd581ff5487ce20668ff776588c"}
Nov 02 12:03:02.527355 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:02.527289    4942 scope.go:122] "RemoveContainer" containerID="4f66532cd52b9a294b09c4595cb30779b719782752ca2458576a2082b47f39fa"
Nov 02 12:03:02.537484 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:02.537465    4942 scope.go:122] "RemoveContainer" containerID="4f66532cd52b9a294b09c4595cb30779b719782752ca2458576a2082b47f39fa"
Nov 02 12:03:02.537737 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:03:02.537717    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f66532cd52b9a294b09c4595cb30779b719782752ca2458576a2082b47f39fa\": container with ID starting with 4f66532cd52b9a294b09c4595cb30779b719782752ca2458576a2082b47f39fa not found: ID does not exist" containerID="4f66532cd52b9a294b09c4595cb30779b719782752ca2458576a2082b47f39fa"
Nov 02 12:03:02.537799 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:02.537749    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f66532cd52b9a294b09c4595cb30779b719782752ca2458576a2082b47f39fa"} err="failed to get container status \"4f66532cd52b9a294b09c4595cb30779b719782752ca2458576a2082b47f39fa\": rpc error: code = NotFound desc = could not find container \"4f66532cd52b9a294b09c4595cb30779b719782752ca2458576a2082b47f39fa\": container with ID starting with 4f66532cd52b9a294b09c4595cb30779b719782752ca2458576a2082b47f39fa not found: ID does not exist"
Nov 02 12:03:02.689908 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:02.689836    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"]
Nov 02 12:03:02.694248 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:02.694219    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-zw4g2"]
Nov 02 12:03:03.540567 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:03.540537    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c9b2ae98-816e-453f-b7cd-b4122d871910" path="/var/lib/kubelet/pods/c9b2ae98-816e-453f-b7cd-b4122d871910/volumes"
Nov 02 12:03:07.455828 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.455795    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-md7fh"]
Nov 02 12:03:07.456227 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.456161    4942 cpu_manager.go:414] "containerMap: removing container" podUID="c9b2ae98-816e-453f-b7cd-b4122d871910" containerName="limitador"
Nov 02 12:03:07.456227 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.456172    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c9b2ae98-816e-453f-b7cd-b4122d871910" containerName="limitador"
Nov 02 12:03:07.456227 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.456197    4942 cpu_manager.go:414] "containerMap: removing container" podUID="39437d8e-b528-46d5-ba95-a18ab968214b" containerName="authorino"
Nov 02 12:03:07.456227 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.456203    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="39437d8e-b528-46d5-ba95-a18ab968214b" containerName="authorino"
Nov 02 12:03:07.456407 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.456273    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="39437d8e-b528-46d5-ba95-a18ab968214b" containerName="authorino"
Nov 02 12:03:07.456407 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.456286    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="c9b2ae98-816e-453f-b7cd-b4122d871910" containerName="limitador"
Nov 02 12:03:07.458615 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.458598    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-md7fh"
Nov 02 12:03:07.513261 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.513236    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 02 12:03:07.533166 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.533135    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/783954f3-441b-40cb-aa26-219ed6d52aa7-data\") pod \"postgres-7cdbf6dc44-md7fh\" (UID: \"783954f3-441b-40cb-aa26-219ed6d52aa7\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-md7fh"
Nov 02 12:03:07.533320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.533264    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wj27\" (UniqueName: \"kubernetes.io/projected/783954f3-441b-40cb-aa26-219ed6d52aa7-kube-api-access-9wj27\") pod \"postgres-7cdbf6dc44-md7fh\" (UID: \"783954f3-441b-40cb-aa26-219ed6d52aa7\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-md7fh"
Nov 02 12:03:07.560783 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.560757    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Nov 02 12:03:07.560930 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.560911    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-qtmgl\""
Nov 02 12:03:07.562755 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.562735    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-md7fh"]
Nov 02 12:03:07.611560 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.611537    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 02 12:03:07.634075 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.634053    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/783954f3-441b-40cb-aa26-219ed6d52aa7-data\") pod \"postgres-7cdbf6dc44-md7fh\" (UID: \"783954f3-441b-40cb-aa26-219ed6d52aa7\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-md7fh"
Nov 02 12:03:07.634179 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.634119    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wj27\" (UniqueName: \"kubernetes.io/projected/783954f3-441b-40cb-aa26-219ed6d52aa7-kube-api-access-9wj27\") pod \"postgres-7cdbf6dc44-md7fh\" (UID: \"783954f3-441b-40cb-aa26-219ed6d52aa7\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-md7fh"
Nov 02 12:03:07.634499 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.634479    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/783954f3-441b-40cb-aa26-219ed6d52aa7-data\") pod \"postgres-7cdbf6dc44-md7fh\" (UID: \"783954f3-441b-40cb-aa26-219ed6d52aa7\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-md7fh"
Nov 02 12:03:07.720703 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.720646    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wj27\" (UniqueName: \"kubernetes.io/projected/783954f3-441b-40cb-aa26-219ed6d52aa7-kube-api-access-9wj27\") pod \"postgres-7cdbf6dc44-md7fh\" (UID: \"783954f3-441b-40cb-aa26-219ed6d52aa7\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-md7fh"
Nov 02 12:03:07.769807 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.769784    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-md7fh"
Nov 02 12:03:07.857387 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.857357    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"]
Nov 02 12:03:07.860585 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.860559    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"
Nov 02 12:03:07.913477 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.913449    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-7ns4r\""
Nov 02 12:03:07.913943 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.913929    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 02 12:03:07.913990 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.913954    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 02 12:03:07.936655 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.936629    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3e1f269e-665c-4850-a92e-3fc1604c6efb-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-t5hm9\" (UID: \"3e1f269e-665c-4850-a92e-3fc1604c6efb\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"
Nov 02 12:03:07.936773 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.936742    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3e1f269e-665c-4850-a92e-3fc1604c6efb-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-t5hm9\" (UID: \"3e1f269e-665c-4850-a92e-3fc1604c6efb\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"
Nov 02 12:03:07.936825 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.936809    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqg9c\" (UniqueName: \"kubernetes.io/projected/3e1f269e-665c-4850-a92e-3fc1604c6efb-kube-api-access-jqg9c\") pod \"kserve-module-controller-manager-7566cf5fc8-t5hm9\" (UID: \"3e1f269e-665c-4850-a92e-3fc1604c6efb\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"
Nov 02 12:03:07.936873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:07.936841    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e1f269e-665c-4850-a92e-3fc1604c6efb-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-t5hm9\" (UID: \"3e1f269e-665c-4850-a92e-3fc1604c6efb\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"
Nov 02 12:03:08.013259 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:08.013194    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Nov 02 12:03:08.015382 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:08.015362    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"]
Nov 02 12:03:08.037923 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:08.037898    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqg9c\" (UniqueName: \"kubernetes.io/projected/3e1f269e-665c-4850-a92e-3fc1604c6efb-kube-api-access-jqg9c\") pod \"kserve-module-controller-manager-7566cf5fc8-t5hm9\" (UID: \"3e1f269e-665c-4850-a92e-3fc1604c6efb\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"
Nov 02 12:03:08.038028 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:08.037934    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e1f269e-665c-4850-a92e-3fc1604c6efb-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-t5hm9\" (UID: \"3e1f269e-665c-4850-a92e-3fc1604c6efb\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"
Nov 02 12:03:08.038028 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:08.037962    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3e1f269e-665c-4850-a92e-3fc1604c6efb-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-t5hm9\" (UID: \"3e1f269e-665c-4850-a92e-3fc1604c6efb\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"
Nov 02 12:03:08.038109 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:08.038035    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3e1f269e-665c-4850-a92e-3fc1604c6efb-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-t5hm9\" (UID: \"3e1f269e-665c-4850-a92e-3fc1604c6efb\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"
Nov 02 12:03:08.038279 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:08.038262    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e1f269e-665c-4850-a92e-3fc1604c6efb-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-t5hm9\" (UID: \"3e1f269e-665c-4850-a92e-3fc1604c6efb\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"
Nov 02 12:03:08.038384 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:08.038364    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3e1f269e-665c-4850-a92e-3fc1604c6efb-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-t5hm9\" (UID: \"3e1f269e-665c-4850-a92e-3fc1604c6efb\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"
Nov 02 12:03:08.038925 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:08.038908    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3e1f269e-665c-4850-a92e-3fc1604c6efb-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-t5hm9\" (UID: \"3e1f269e-665c-4850-a92e-3fc1604c6efb\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"
Nov 02 12:03:08.098870 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:08.098841    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqg9c\" (UniqueName: \"kubernetes.io/projected/3e1f269e-665c-4850-a92e-3fc1604c6efb-kube-api-access-jqg9c\") pod \"kserve-module-controller-manager-7566cf5fc8-t5hm9\" (UID: \"3e1f269e-665c-4850-a92e-3fc1604c6efb\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"
Nov 02 12:03:08.139557 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:08.139511    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-md7fh"]
Nov 02 12:03:08.140915 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:03:08.140883    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod783954f3_441b_40cb_aa26_219ed6d52aa7.slice/crio-8dfbe98ceefcc4920873ed0a67ba224b043d312171637690edfdb249b7e8d18c WatchSource:0}: Error finding container 8dfbe98ceefcc4920873ed0a67ba224b043d312171637690edfdb249b7e8d18c: Status 404 returned error can't find the container with id 8dfbe98ceefcc4920873ed0a67ba224b043d312171637690edfdb249b7e8d18c
Nov 02 12:03:08.171698 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:08.171676    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"
Nov 02 12:03:08.562116 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:08.562081    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-md7fh" event={"ID":"783954f3-441b-40cb-aa26-219ed6d52aa7","Type":"ContainerStarted","Data":"8dfbe98ceefcc4920873ed0a67ba224b043d312171637690edfdb249b7e8d18c"}
Nov 02 12:03:08.594756 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:08.594723    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"]
Nov 02 12:03:08.600438 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:03:08.600406    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e1f269e_665c_4850_a92e_3fc1604c6efb.slice/crio-610fbed197d0226d04b5752b226b01c2f4203ed0ce770814e7be3ed5ab2b4288 WatchSource:0}: Error finding container 610fbed197d0226d04b5752b226b01c2f4203ed0ce770814e7be3ed5ab2b4288: Status 404 returned error can't find the container with id 610fbed197d0226d04b5752b226b01c2f4203ed0ce770814e7be3ed5ab2b4288
Nov 02 12:03:09.567735 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:09.567694    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9" event={"ID":"3e1f269e-665c-4850-a92e-3fc1604c6efb","Type":"ContainerStarted","Data":"610fbed197d0226d04b5752b226b01c2f4203ed0ce770814e7be3ed5ab2b4288"}
Nov 02 12:03:13.598246 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:13.598215    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9" event={"ID":"3e1f269e-665c-4850-a92e-3fc1604c6efb","Type":"ContainerStarted","Data":"5993ea6acc82bfc8f13797a301831a17e7a75d6677a438b234b45fe55f9043af"}
Nov 02 12:03:13.598672 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:13.598483    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"
Nov 02 12:03:13.782380 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:13.782291    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9" podStartSLOduration=2.418551624 podStartE2EDuration="6.782269669s" totalImagesPullingTime="4.363718045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:03:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:03:13.779629094 +0000 UTC m=+420.350978852" watchObservedRunningTime="2026-11-02 12:03:13.782269669 +0000 UTC m=+420.353619418"
Nov 02 12:03:14.771531 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:14.771504    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Nov 02 12:03:15.608836 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:15.608797    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-md7fh" event={"ID":"783954f3-441b-40cb-aa26-219ed6d52aa7","Type":"ContainerStarted","Data":"80c46a1e1f1af730f911b4e18185093a17a1a08d9c7083d0ad12729202853abc"}
Nov 02 12:03:15.608997 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:15.608948    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-md7fh"
Nov 02 12:03:17.164681 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.164626    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-md7fh" podStartSLOduration=3.629092653 podStartE2EDuration="10.164608535s" totalImagesPullingTime="6.535515882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:03:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:03:15.796467228 +0000 UTC m=+422.367816985" watchObservedRunningTime="2026-11-02 12:03:17.164608535 +0000 UTC m=+423.735958337"
Nov 02 12:03:17.165321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.165281    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"]
Nov 02 12:03:17.168649 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.168633    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"
Nov 02 12:03:17.221475 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.221450    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Nov 02 12:03:17.221475 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.221450    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-hv7pq\""
Nov 02 12:03:17.221646 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.221533    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Nov 02 12:03:17.231320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.230679    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p28nd\" (UniqueName: \"kubernetes.io/projected/19141d30-5c46-4fd0-ad55-c5f6f9e014f5-kube-api-access-p28nd\") pod \"jobset-controller-manager-7dcb89589c-48jk8\" (UID: \"19141d30-5c46-4fd0-ad55-c5f6f9e014f5\") " pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"
Nov 02 12:03:17.231320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.230741    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/19141d30-5c46-4fd0-ad55-c5f6f9e014f5-manager-config\") pod \"jobset-controller-manager-7dcb89589c-48jk8\" (UID: \"19141d30-5c46-4fd0-ad55-c5f6f9e014f5\") " pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"
Nov 02 12:03:17.231320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.230815    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19141d30-5c46-4fd0-ad55-c5f6f9e014f5-cert\") pod \"jobset-controller-manager-7dcb89589c-48jk8\" (UID: \"19141d30-5c46-4fd0-ad55-c5f6f9e014f5\") " pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"
Nov 02 12:03:17.231320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.230876    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19141d30-5c46-4fd0-ad55-c5f6f9e014f5-metrics-certs\") pod \"jobset-controller-manager-7dcb89589c-48jk8\" (UID: \"19141d30-5c46-4fd0-ad55-c5f6f9e014f5\") " pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"
Nov 02 12:03:17.270841 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.270810    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 02 12:03:17.273047 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.273026    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"]
Nov 02 12:03:17.321676 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.321652    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 02 12:03:17.321787 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.321741    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Nov 02 12:03:17.332286 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.332266    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19141d30-5c46-4fd0-ad55-c5f6f9e014f5-cert\") pod \"jobset-controller-manager-7dcb89589c-48jk8\" (UID: \"19141d30-5c46-4fd0-ad55-c5f6f9e014f5\") " pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"
Nov 02 12:03:17.332389 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.332333    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19141d30-5c46-4fd0-ad55-c5f6f9e014f5-metrics-certs\") pod \"jobset-controller-manager-7dcb89589c-48jk8\" (UID: \"19141d30-5c46-4fd0-ad55-c5f6f9e014f5\") " pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"
Nov 02 12:03:17.332389 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.332383    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p28nd\" (UniqueName: \"kubernetes.io/projected/19141d30-5c46-4fd0-ad55-c5f6f9e014f5-kube-api-access-p28nd\") pod \"jobset-controller-manager-7dcb89589c-48jk8\" (UID: \"19141d30-5c46-4fd0-ad55-c5f6f9e014f5\") " pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"
Nov 02 12:03:17.332501 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.332409    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/19141d30-5c46-4fd0-ad55-c5f6f9e014f5-manager-config\") pod \"jobset-controller-manager-7dcb89589c-48jk8\" (UID: \"19141d30-5c46-4fd0-ad55-c5f6f9e014f5\") " pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"
Nov 02 12:03:17.333105 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.333086    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/19141d30-5c46-4fd0-ad55-c5f6f9e014f5-manager-config\") pod \"jobset-controller-manager-7dcb89589c-48jk8\" (UID: \"19141d30-5c46-4fd0-ad55-c5f6f9e014f5\") " pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"
Nov 02 12:03:17.335011 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.334988    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/19141d30-5c46-4fd0-ad55-c5f6f9e014f5-cert\") pod \"jobset-controller-manager-7dcb89589c-48jk8\" (UID: \"19141d30-5c46-4fd0-ad55-c5f6f9e014f5\") " pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"
Nov 02 12:03:17.335619 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.335600    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19141d30-5c46-4fd0-ad55-c5f6f9e014f5-metrics-certs\") pod \"jobset-controller-manager-7dcb89589c-48jk8\" (UID: \"19141d30-5c46-4fd0-ad55-c5f6f9e014f5\") " pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"
Nov 02 12:03:17.392455 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.392433    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p28nd\" (UniqueName: \"kubernetes.io/projected/19141d30-5c46-4fd0-ad55-c5f6f9e014f5-kube-api-access-p28nd\") pod \"jobset-controller-manager-7dcb89589c-48jk8\" (UID: \"19141d30-5c46-4fd0-ad55-c5f6f9e014f5\") " pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"
Nov 02 12:03:17.480912 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.480853    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"
Nov 02 12:03:17.853227 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:17.853206    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"]
Nov 02 12:03:17.855235 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:03:17.855214    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19141d30_5c46_4fd0_ad55_c5f6f9e014f5.slice/crio-65d1e1824b571d6a980a316509b1c802ed0271caedee7b48e5e1b8edada96cdb WatchSource:0}: Error finding container 65d1e1824b571d6a980a316509b1c802ed0271caedee7b48e5e1b8edada96cdb: Status 404 returned error can't find the container with id 65d1e1824b571d6a980a316509b1c802ed0271caedee7b48e5e1b8edada96cdb
Nov 02 12:03:18.623557 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:18.623519    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8" event={"ID":"19141d30-5c46-4fd0-ad55-c5f6f9e014f5","Type":"ContainerStarted","Data":"65d1e1824b571d6a980a316509b1c802ed0271caedee7b48e5e1b8edada96cdb"}
Nov 02 12:03:21.651674 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:21.651641    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-md7fh"
Nov 02 12:03:22.647554 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:22.647519    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8" event={"ID":"19141d30-5c46-4fd0-ad55-c5f6f9e014f5","Type":"ContainerStarted","Data":"2ecdaa85367e7325c31a5200b994998cd947a70bd4eaab5eb42e7335d71ef2dd"}
Nov 02 12:03:22.647735 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:22.647611    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"
Nov 02 12:03:22.764746 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:22.764688    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8" podStartSLOduration=1.885060594 podStartE2EDuration="5.764672401s" totalImagesPullingTime="3.879611807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:03:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:03:22.763186746 +0000 UTC m=+429.334536514" watchObservedRunningTime="2026-11-02 12:03:22.764672401 +0000 UTC m=+429.336022159"
Nov 02 12:03:30.982690 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:03:30.982647    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:03:33.657100 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:33.657071    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-7dcb89589c-48jk8"
Nov 02 12:03:34.604378 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:03:34.604345    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-t5hm9"
Nov 02 12:04:00.982366 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:04:00.982325    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:04:23.969544 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:23.969438    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-mg624 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 12:04:23.969544 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:23.969517    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624" podUID="9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:04:25.959009 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:25.958970    4942 generic.go:356] "Generic (PLEG): container finished" podID="9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" containerID="de2beb053e81c92e915186a39876f11c76d59c750d59ce755b3f3a9143a8ebf4" exitCode=0
Nov 02 12:04:25.959507 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:25.959042    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624" event={"ID":"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42","Type":"ContainerDied","Data":"de2beb053e81c92e915186a39876f11c76d59c750d59ce755b3f3a9143a8ebf4"}
Nov 02 12:04:25.959507 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:25.959416    4942 scope.go:122] "RemoveContainer" containerID="de2beb053e81c92e915186a39876f11c76d59c750d59ce755b3f3a9143a8ebf4"
Nov 02 12:04:25.960470 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:25.960441    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:04:25.960529 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:25.960510    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:04:25.960562 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:25.960542    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:04:26.983146 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:26.983109    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624" event={"ID":"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42","Type":"ContainerStarted","Data":"fedc2ed2418fc64d4a44a13b5a204838132c812645d2309012647e3ddcfbcf8c"}
Nov 02 12:04:27.977324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:27.974691    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:04:27.977324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:27.974813    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-mg624 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Nov 02 12:04:27.977324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:27.974855    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624" podUID="9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Nov 02 12:04:28.880938 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:28.880900    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-pkkjq"]
Nov 02 12:04:28.885867 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:28.885848    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-pkkjq"
Nov 02 12:04:28.937816 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:28.937789    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-pkkjq"]
Nov 02 12:04:28.951198 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:28.951174    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv96h\" (UniqueName: \"kubernetes.io/projected/8b77bfb9-b094-4f9e-a9b0-8cf079102853-kube-api-access-sv96h\") pod \"authorino-6f6944bcdd-pkkjq\" (UID: \"8b77bfb9-b094-4f9e-a9b0-8cf079102853\") " pod="kuadrant-system/authorino-6f6944bcdd-pkkjq"
Nov 02 12:04:28.978001 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:28.977956    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-mg624 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Nov 02 12:04:28.978171 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:28.978052    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624" podUID="9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Nov 02 12:04:29.073858 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.073819    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv96h\" (UniqueName: \"kubernetes.io/projected/8b77bfb9-b094-4f9e-a9b0-8cf079102853-kube-api-access-sv96h\") pod \"authorino-6f6944bcdd-pkkjq\" (UID: \"8b77bfb9-b094-4f9e-a9b0-8cf079102853\") " pod="kuadrant-system/authorino-6f6944bcdd-pkkjq"
Nov 02 12:04:29.137011 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.136943    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv96h\" (UniqueName: \"kubernetes.io/projected/8b77bfb9-b094-4f9e-a9b0-8cf079102853-kube-api-access-sv96h\") pod \"authorino-6f6944bcdd-pkkjq\" (UID: \"8b77bfb9-b094-4f9e-a9b0-8cf079102853\") " pod="kuadrant-system/authorino-6f6944bcdd-pkkjq"
Nov 02 12:04:29.197704 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.197672    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-pkkjq"
Nov 02 12:04:29.280550 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.274881    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-pkkjq"]
Nov 02 12:04:29.287277 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.287242    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c76cf7d7c-wlvz7"]
Nov 02 12:04:29.291196 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.291173    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c76cf7d7c-wlvz7"
Nov 02 12:04:29.344217 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.344183    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c76cf7d7c-wlvz7"]
Nov 02 12:04:29.384791 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.384756    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc457\" (UniqueName: \"kubernetes.io/projected/ad2cda63-eeec-4cdf-b1b5-71df3af11d9a-kube-api-access-pc457\") pod \"authorino-c76cf7d7c-wlvz7\" (UID: \"ad2cda63-eeec-4cdf-b1b5-71df3af11d9a\") " pod="kuadrant-system/authorino-c76cf7d7c-wlvz7"
Nov 02 12:04:29.485859 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.485783    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc457\" (UniqueName: \"kubernetes.io/projected/ad2cda63-eeec-4cdf-b1b5-71df3af11d9a-kube-api-access-pc457\") pod \"authorino-c76cf7d7c-wlvz7\" (UID: \"ad2cda63-eeec-4cdf-b1b5-71df3af11d9a\") " pod="kuadrant-system/authorino-c76cf7d7c-wlvz7"
Nov 02 12:04:29.546594 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.546565    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc457\" (UniqueName: \"kubernetes.io/projected/ad2cda63-eeec-4cdf-b1b5-71df3af11d9a-kube-api-access-pc457\") pod \"authorino-c76cf7d7c-wlvz7\" (UID: \"ad2cda63-eeec-4cdf-b1b5-71df3af11d9a\") " pod="kuadrant-system/authorino-c76cf7d7c-wlvz7"
Nov 02 12:04:29.596097 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.596063    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-pkkjq"]
Nov 02 12:04:29.597863 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:04:29.597828    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b77bfb9_b094_4f9e_a9b0_8cf079102853.slice/crio-e8705eefe69cd5157c0d0574ee740209009ba5a03d38086e48f611a99e8af672 WatchSource:0}: Error finding container e8705eefe69cd5157c0d0574ee740209009ba5a03d38086e48f611a99e8af672: Status 404 returned error can't find the container with id e8705eefe69cd5157c0d0574ee740209009ba5a03d38086e48f611a99e8af672
Nov 02 12:04:29.606073 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.606053    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c76cf7d7c-wlvz7"
Nov 02 12:04:29.713001 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.712975    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c76cf7d7c-wlvz7"]
Nov 02 12:04:29.779359 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.779259    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-bc9d986f5-bqcdf"]
Nov 02 12:04:29.783974 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.783958    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bc9d986f5-bqcdf"
Nov 02 12:04:29.820108 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.820081    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bc9d986f5-bqcdf"]
Nov 02 12:04:29.836431 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.836410    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Nov 02 12:04:29.890677 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.890648    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ea0a699-2c5b-46f0-8141-bb9a1396d76d-tls-cert\") pod \"authorino-bc9d986f5-bqcdf\" (UID: \"8ea0a699-2c5b-46f0-8141-bb9a1396d76d\") " pod="kuadrant-system/authorino-bc9d986f5-bqcdf"
Nov 02 12:04:29.891025 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.890699    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztdwv\" (UniqueName: \"kubernetes.io/projected/8ea0a699-2c5b-46f0-8141-bb9a1396d76d-kube-api-access-ztdwv\") pod \"authorino-bc9d986f5-bqcdf\" (UID: \"8ea0a699-2c5b-46f0-8141-bb9a1396d76d\") " pod="kuadrant-system/authorino-bc9d986f5-bqcdf"
Nov 02 12:04:29.972322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.970725    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-mg624 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Nov 02 12:04:29.972322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.970788    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624" podUID="9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Nov 02 12:04:29.991956 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.991931    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ea0a699-2c5b-46f0-8141-bb9a1396d76d-tls-cert\") pod \"authorino-bc9d986f5-bqcdf\" (UID: \"8ea0a699-2c5b-46f0-8141-bb9a1396d76d\") " pod="kuadrant-system/authorino-bc9d986f5-bqcdf"
Nov 02 12:04:29.992094 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.991988    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztdwv\" (UniqueName: \"kubernetes.io/projected/8ea0a699-2c5b-46f0-8141-bb9a1396d76d-kube-api-access-ztdwv\") pod \"authorino-bc9d986f5-bqcdf\" (UID: \"8ea0a699-2c5b-46f0-8141-bb9a1396d76d\") " pod="kuadrant-system/authorino-bc9d986f5-bqcdf"
Nov 02 12:04:29.994781 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.994751    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ea0a699-2c5b-46f0-8141-bb9a1396d76d-tls-cert\") pod \"authorino-bc9d986f5-bqcdf\" (UID: \"8ea0a699-2c5b-46f0-8141-bb9a1396d76d\") " pod="kuadrant-system/authorino-bc9d986f5-bqcdf"
Nov 02 12:04:29.996983 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.996144    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c76cf7d7c-wlvz7"]
Nov 02 12:04:29.997931 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:04:29.997900    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad2cda63_eeec_4cdf_b1b5_71df3af11d9a.slice/crio-0e870acb9cecb0995269b7fab23ba656185cfaaee3ccefd15b102f601399d115 WatchSource:0}: Error finding container 0e870acb9cecb0995269b7fab23ba656185cfaaee3ccefd15b102f601399d115: Status 404 returned error can't find the container with id 0e870acb9cecb0995269b7fab23ba656185cfaaee3ccefd15b102f601399d115
Nov 02 12:04:29.998819 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:29.998792    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-pkkjq" event={"ID":"8b77bfb9-b094-4f9e-a9b0-8cf079102853","Type":"ContainerStarted","Data":"e8705eefe69cd5157c0d0574ee740209009ba5a03d38086e48f611a99e8af672"}
Nov 02 12:04:30.077378 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:30.076982    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztdwv\" (UniqueName: \"kubernetes.io/projected/8ea0a699-2c5b-46f0-8141-bb9a1396d76d-kube-api-access-ztdwv\") pod \"authorino-bc9d986f5-bqcdf\" (UID: \"8ea0a699-2c5b-46f0-8141-bb9a1396d76d\") " pod="kuadrant-system/authorino-bc9d986f5-bqcdf"
Nov 02 12:04:30.096021 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:30.095994    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bc9d986f5-bqcdf"
Nov 02 12:04:30.477894 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:30.476907    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bc9d986f5-bqcdf"]
Nov 02 12:04:30.480017 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:04:30.479990    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ea0a699_2c5b_46f0_8141_bb9a1396d76d.slice/crio-375a28539d97ba7ef220c192055ec41c6272d5c6090e5455c7741868da2d9168 WatchSource:0}: Error finding container 375a28539d97ba7ef220c192055ec41c6272d5c6090e5455c7741868da2d9168: Status 404 returned error can't find the container with id 375a28539d97ba7ef220c192055ec41c6272d5c6090e5455c7741868da2d9168
Nov 02 12:04:30.976812 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:30.976778    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-mg624 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Nov 02 12:04:30.977414 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:30.977377    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624" podUID="9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Nov 02 12:04:31.004813 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:04:31.004778    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:04:31.005243 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:31.005222    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bc9d986f5-bqcdf" event={"ID":"8ea0a699-2c5b-46f0-8141-bb9a1396d76d","Type":"ContainerStarted","Data":"375a28539d97ba7ef220c192055ec41c6272d5c6090e5455c7741868da2d9168"}
Nov 02 12:04:31.006440 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:31.006415    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c76cf7d7c-wlvz7" event={"ID":"ad2cda63-eeec-4cdf-b1b5-71df3af11d9a","Type":"ContainerStarted","Data":"0e870acb9cecb0995269b7fab23ba656185cfaaee3ccefd15b102f601399d115"}
Nov 02 12:04:32.019507 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.018676    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c76cf7d7c-wlvz7" event={"ID":"ad2cda63-eeec-4cdf-b1b5-71df3af11d9a","Type":"ContainerStarted","Data":"2580912c653009335ccb0c6c99e4cc9c24bd32498d682612c15b8b914d90a9fa"}
Nov 02 12:04:32.019507 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.018855    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-c76cf7d7c-wlvz7" podUID="ad2cda63-eeec-4cdf-b1b5-71df3af11d9a" containerName="authorino" containerID="cri-o://2580912c653009335ccb0c6c99e4cc9c24bd32498d682612c15b8b914d90a9fa" gracePeriod=30
Nov 02 12:04:32.024840 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.024582    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-pkkjq" event={"ID":"8b77bfb9-b094-4f9e-a9b0-8cf079102853","Type":"ContainerStarted","Data":"4f61ddd2d0f35f3deaf4570d263f830f475636c9dae5564f753747b842efac6e"}
Nov 02 12:04:32.024840 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.024722    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-6f6944bcdd-pkkjq" podUID="8b77bfb9-b094-4f9e-a9b0-8cf079102853" containerName="authorino" containerID="cri-o://4f61ddd2d0f35f3deaf4570d263f830f475636c9dae5564f753747b842efac6e" gracePeriod=30
Nov 02 12:04:32.031234 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.030689    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bc9d986f5-bqcdf" event={"ID":"8ea0a699-2c5b-46f0-8141-bb9a1396d76d","Type":"ContainerStarted","Data":"f09a0115211083d18aea72109e33ea0dcdec8a1d6fa31d0244f7b5ec2baf373b"}
Nov 02 12:04:32.081103 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.081069    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:04:32.081513 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.081494    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:04:32.082358 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.082335    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:04:32.133802 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.133741    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-c76cf7d7c-wlvz7" podStartSLOduration=1.915017054 podStartE2EDuration="3.133722845s" totalImagesPullingTime="1.218705791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:04:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:04:32.132028192 +0000 UTC m=+498.703377948" watchObservedRunningTime="2026-11-02 12:04:32.133722845 +0000 UTC m=+498.705072602"
Nov 02 12:04:32.344990 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.344963    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-pkkjq"
Nov 02 12:04:32.347207 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.347192    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c76cf7d7c-wlvz7"
Nov 02 12:04:32.369919 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.369858    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-6f6944bcdd-pkkjq" podStartSLOduration=2.829683939 podStartE2EDuration="4.369839226s" totalImagesPullingTime="1.540155287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:04:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:04:32.368475169 +0000 UTC m=+498.939824927" watchObservedRunningTime="2026-11-02 12:04:32.369839226 +0000 UTC m=+498.941188984"
Nov 02 12:04:32.414761 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.414734    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b77bfb9-b094-4f9e-a9b0-8cf079102853-kube-api-access-sv96h\" (UniqueName: \"kubernetes.io/projected/8b77bfb9-b094-4f9e-a9b0-8cf079102853-kube-api-access-sv96h\") pod \"8b77bfb9-b094-4f9e-a9b0-8cf079102853\" (UID: \"8b77bfb9-b094-4f9e-a9b0-8cf079102853\") "
Nov 02 12:04:32.414910 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.414773    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad2cda63-eeec-4cdf-b1b5-71df3af11d9a-kube-api-access-pc457\" (UniqueName: \"kubernetes.io/projected/ad2cda63-eeec-4cdf-b1b5-71df3af11d9a-kube-api-access-pc457\") pod \"ad2cda63-eeec-4cdf-b1b5-71df3af11d9a\" (UID: \"ad2cda63-eeec-4cdf-b1b5-71df3af11d9a\") "
Nov 02 12:04:32.416765 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.416739    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b77bfb9-b094-4f9e-a9b0-8cf079102853-kube-api-access-sv96h" pod "8b77bfb9-b094-4f9e-a9b0-8cf079102853" (UID: "8b77bfb9-b094-4f9e-a9b0-8cf079102853"). InnerVolumeSpecName "kube-api-access-sv96h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:04:32.416937 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.416917    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad2cda63-eeec-4cdf-b1b5-71df3af11d9a-kube-api-access-pc457" pod "ad2cda63-eeec-4cdf-b1b5-71df3af11d9a" (UID: "ad2cda63-eeec-4cdf-b1b5-71df3af11d9a"). InnerVolumeSpecName "kube-api-access-pc457". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:04:32.484227 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.484165    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-bc9d986f5-bqcdf" podStartSLOduration=2.252759155 podStartE2EDuration="3.484149817s" totalImagesPullingTime="1.231390662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:04:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:04:32.483883264 +0000 UTC m=+499.055233021" watchObservedRunningTime="2026-11-02 12:04:32.484149817 +0000 UTC m=+499.055499574"
Nov 02 12:04:32.511347 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.511283    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-vn25l"]
Nov 02 12:04:32.511527 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.511497    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-vn25l" podUID="d2a2667a-f64b-431a-a6d8-b60a67c4df50" containerName="authorino" containerID="cri-o://77c6a7d38e5ebb7e30b6b8a7371d2c853ea08107168e6dc46c433c01d77ff99e" gracePeriod=30
Nov 02 12:04:32.515644 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.515606    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sv96h\" (UniqueName: \"kubernetes.io/projected/8b77bfb9-b094-4f9e-a9b0-8cf079102853-kube-api-access-sv96h\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:04:32.515644 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.515646    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pc457\" (UniqueName: \"kubernetes.io/projected/ad2cda63-eeec-4cdf-b1b5-71df3af11d9a-kube-api-access-pc457\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:04:32.762701 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.762678    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-vn25l"
Nov 02 12:04:32.818264 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.818231    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d2a2667a-f64b-431a-a6d8-b60a67c4df50-kube-api-access-d4wxh\" (UniqueName: \"kubernetes.io/projected/d2a2667a-f64b-431a-a6d8-b60a67c4df50-kube-api-access-d4wxh\") pod \"d2a2667a-f64b-431a-a6d8-b60a67c4df50\" (UID: \"d2a2667a-f64b-431a-a6d8-b60a67c4df50\") "
Nov 02 12:04:32.820359 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.820332    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2a2667a-f64b-431a-a6d8-b60a67c4df50-kube-api-access-d4wxh" pod "d2a2667a-f64b-431a-a6d8-b60a67c4df50" (UID: "d2a2667a-f64b-431a-a6d8-b60a67c4df50"). InnerVolumeSpecName "kube-api-access-d4wxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:04:32.919749 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:32.919674    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4wxh\" (UniqueName: \"kubernetes.io/projected/d2a2667a-f64b-431a-a6d8-b60a67c4df50-kube-api-access-d4wxh\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:04:33.038379 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.038340    4942 generic.go:356] "Generic (PLEG): container finished" podID="ad2cda63-eeec-4cdf-b1b5-71df3af11d9a" containerID="2580912c653009335ccb0c6c99e4cc9c24bd32498d682612c15b8b914d90a9fa" exitCode=0
Nov 02 12:04:33.038830 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.038396    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c76cf7d7c-wlvz7"
Nov 02 12:04:33.038830 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.038432    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c76cf7d7c-wlvz7" event={"ID":"ad2cda63-eeec-4cdf-b1b5-71df3af11d9a","Type":"ContainerDied","Data":"2580912c653009335ccb0c6c99e4cc9c24bd32498d682612c15b8b914d90a9fa"}
Nov 02 12:04:33.038830 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.038472    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c76cf7d7c-wlvz7" event={"ID":"ad2cda63-eeec-4cdf-b1b5-71df3af11d9a","Type":"ContainerDied","Data":"0e870acb9cecb0995269b7fab23ba656185cfaaee3ccefd15b102f601399d115"}
Nov 02 12:04:33.038830 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.038494    4942 scope.go:122] "RemoveContainer" containerID="2580912c653009335ccb0c6c99e4cc9c24bd32498d682612c15b8b914d90a9fa"
Nov 02 12:04:33.039910 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.039882    4942 generic.go:356] "Generic (PLEG): container finished" podID="8b77bfb9-b094-4f9e-a9b0-8cf079102853" containerID="4f61ddd2d0f35f3deaf4570d263f830f475636c9dae5564f753747b842efac6e" exitCode=0
Nov 02 12:04:33.040015 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.039930    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-pkkjq"
Nov 02 12:04:33.040015 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.039972    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-pkkjq" event={"ID":"8b77bfb9-b094-4f9e-a9b0-8cf079102853","Type":"ContainerDied","Data":"4f61ddd2d0f35f3deaf4570d263f830f475636c9dae5564f753747b842efac6e"}
Nov 02 12:04:33.040117 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.040015    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-pkkjq" event={"ID":"8b77bfb9-b094-4f9e-a9b0-8cf079102853","Type":"ContainerDied","Data":"e8705eefe69cd5157c0d0574ee740209009ba5a03d38086e48f611a99e8af672"}
Nov 02 12:04:33.041391 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.041365    4942 generic.go:356] "Generic (PLEG): container finished" podID="d2a2667a-f64b-431a-a6d8-b60a67c4df50" containerID="77c6a7d38e5ebb7e30b6b8a7371d2c853ea08107168e6dc46c433c01d77ff99e" exitCode=0
Nov 02 12:04:33.041468 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.041405    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-vn25l"
Nov 02 12:04:33.041468 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.041458    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-vn25l" event={"ID":"d2a2667a-f64b-431a-a6d8-b60a67c4df50","Type":"ContainerDied","Data":"77c6a7d38e5ebb7e30b6b8a7371d2c853ea08107168e6dc46c433c01d77ff99e"}
Nov 02 12:04:33.041539 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.041490    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-vn25l" event={"ID":"d2a2667a-f64b-431a-a6d8-b60a67c4df50","Type":"ContainerDied","Data":"32a361bea633ff129942a9be93d3922f5e23c9987e6b032b538b53dcf3d27a54"}
Nov 02 12:04:33.047469 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.047435    4942 scope.go:122] "RemoveContainer" containerID="2580912c653009335ccb0c6c99e4cc9c24bd32498d682612c15b8b914d90a9fa"
Nov 02 12:04:33.047825 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:04:33.047803    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2580912c653009335ccb0c6c99e4cc9c24bd32498d682612c15b8b914d90a9fa\": container with ID starting with 2580912c653009335ccb0c6c99e4cc9c24bd32498d682612c15b8b914d90a9fa not found: ID does not exist" containerID="2580912c653009335ccb0c6c99e4cc9c24bd32498d682612c15b8b914d90a9fa"
Nov 02 12:04:33.047909 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.047835    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2580912c653009335ccb0c6c99e4cc9c24bd32498d682612c15b8b914d90a9fa"} err="failed to get container status \"2580912c653009335ccb0c6c99e4cc9c24bd32498d682612c15b8b914d90a9fa\": rpc error: code = NotFound desc = could not find container \"2580912c653009335ccb0c6c99e4cc9c24bd32498d682612c15b8b914d90a9fa\": container with ID starting with 2580912c653009335ccb0c6c99e4cc9c24bd32498d682612c15b8b914d90a9fa not found: ID does not exist"
Nov 02 12:04:33.047909 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.047858    4942 scope.go:122] "RemoveContainer" containerID="4f61ddd2d0f35f3deaf4570d263f830f475636c9dae5564f753747b842efac6e"
Nov 02 12:04:33.057500 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.057482    4942 scope.go:122] "RemoveContainer" containerID="4f61ddd2d0f35f3deaf4570d263f830f475636c9dae5564f753747b842efac6e"
Nov 02 12:04:33.057740 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:04:33.057719    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f61ddd2d0f35f3deaf4570d263f830f475636c9dae5564f753747b842efac6e\": container with ID starting with 4f61ddd2d0f35f3deaf4570d263f830f475636c9dae5564f753747b842efac6e not found: ID does not exist" containerID="4f61ddd2d0f35f3deaf4570d263f830f475636c9dae5564f753747b842efac6e"
Nov 02 12:04:33.057780 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.057748    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f61ddd2d0f35f3deaf4570d263f830f475636c9dae5564f753747b842efac6e"} err="failed to get container status \"4f61ddd2d0f35f3deaf4570d263f830f475636c9dae5564f753747b842efac6e\": rpc error: code = NotFound desc = could not find container \"4f61ddd2d0f35f3deaf4570d263f830f475636c9dae5564f753747b842efac6e\": container with ID starting with 4f61ddd2d0f35f3deaf4570d263f830f475636c9dae5564f753747b842efac6e not found: ID does not exist"
Nov 02 12:04:33.057780 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.057765    4942 scope.go:122] "RemoveContainer" containerID="77c6a7d38e5ebb7e30b6b8a7371d2c853ea08107168e6dc46c433c01d77ff99e"
Nov 02 12:04:33.065149 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.065127    4942 scope.go:122] "RemoveContainer" containerID="77c6a7d38e5ebb7e30b6b8a7371d2c853ea08107168e6dc46c433c01d77ff99e"
Nov 02 12:04:33.065470 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:04:33.065452    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77c6a7d38e5ebb7e30b6b8a7371d2c853ea08107168e6dc46c433c01d77ff99e\": container with ID starting with 77c6a7d38e5ebb7e30b6b8a7371d2c853ea08107168e6dc46c433c01d77ff99e not found: ID does not exist" containerID="77c6a7d38e5ebb7e30b6b8a7371d2c853ea08107168e6dc46c433c01d77ff99e"
Nov 02 12:04:33.065544 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.065475    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77c6a7d38e5ebb7e30b6b8a7371d2c853ea08107168e6dc46c433c01d77ff99e"} err="failed to get container status \"77c6a7d38e5ebb7e30b6b8a7371d2c853ea08107168e6dc46c433c01d77ff99e\": rpc error: code = NotFound desc = could not find container \"77c6a7d38e5ebb7e30b6b8a7371d2c853ea08107168e6dc46c433c01d77ff99e\": container with ID starting with 77c6a7d38e5ebb7e30b6b8a7371d2c853ea08107168e6dc46c433c01d77ff99e not found: ID does not exist"
Nov 02 12:04:33.211987 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.207525    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c76cf7d7c-wlvz7"]
Nov 02 12:04:33.214953 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.214923    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c76cf7d7c-wlvz7"]
Nov 02 12:04:33.324512 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.324477    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-vn25l"]
Nov 02 12:04:33.327950 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.327924    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-vn25l"]
Nov 02 12:04:33.436324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.436276    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-pkkjq"]
Nov 02 12:04:33.438089 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.438067    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-pkkjq"]
Nov 02 12:04:33.544023 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.543948    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b77bfb9-b094-4f9e-a9b0-8cf079102853" path="/var/lib/kubelet/pods/8b77bfb9-b094-4f9e-a9b0-8cf079102853/volumes"
Nov 02 12:04:33.544290 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.544276    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad2cda63-eeec-4cdf-b1b5-71df3af11d9a" path="/var/lib/kubelet/pods/ad2cda63-eeec-4cdf-b1b5-71df3af11d9a/volumes"
Nov 02 12:04:33.544584 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:04:33.544572    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d2a2667a-f64b-431a-a6d8-b60a67c4df50" path="/var/lib/kubelet/pods/d2a2667a-f64b-431a-a6d8-b60a67c4df50/volumes"
Nov 02 12:05:00.982595 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:05:00.982559    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:05:30.982517 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:05:30.982472    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:06:00.981937 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:06:00.981846    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:06:13.599139 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:06:13.599109    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:06:13.600460 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:06:13.600435    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:06:30.982007 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:06:30.981952    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:07:00.983256 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:07:00.983222    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:07:04.281098 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.281058    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"]
Nov 02 12:07:04.281516 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.281472    4942 cpu_manager.go:414] "containerMap: removing container" podUID="d2a2667a-f64b-431a-a6d8-b60a67c4df50" containerName="authorino"
Nov 02 12:07:04.281516 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.281485    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d2a2667a-f64b-431a-a6d8-b60a67c4df50" containerName="authorino"
Nov 02 12:07:04.281516 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.281498    4942 cpu_manager.go:414] "containerMap: removing container" podUID="ad2cda63-eeec-4cdf-b1b5-71df3af11d9a" containerName="authorino"
Nov 02 12:07:04.281516 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.281502    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad2cda63-eeec-4cdf-b1b5-71df3af11d9a" containerName="authorino"
Nov 02 12:07:04.281648 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.281531    4942 cpu_manager.go:414] "containerMap: removing container" podUID="8b77bfb9-b094-4f9e-a9b0-8cf079102853" containerName="authorino"
Nov 02 12:07:04.281648 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.281537    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b77bfb9-b094-4f9e-a9b0-8cf079102853" containerName="authorino"
Nov 02 12:07:04.281648 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.281602    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="d2a2667a-f64b-431a-a6d8-b60a67c4df50" containerName="authorino"
Nov 02 12:07:04.281648 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.281610    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad2cda63-eeec-4cdf-b1b5-71df3af11d9a" containerName="authorino"
Nov 02 12:07:04.281648 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.281617    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b77bfb9-b094-4f9e-a9b0-8cf079102853" containerName="authorino"
Nov 02 12:07:04.285069 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.285046    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.337619 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.337579    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 02 12:07:04.377389 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.377353    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ded7be50-33c3-48ec-9c1e-cab1aaf939b1-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-2hcp9\" (UID: \"ded7be50-33c3-48ec-9c1e-cab1aaf939b1\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.377575 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.377403    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ded7be50-33c3-48ec-9c1e-cab1aaf939b1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-2hcp9\" (UID: \"ded7be50-33c3-48ec-9c1e-cab1aaf939b1\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.377575 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.377428    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ded7be50-33c3-48ec-9c1e-cab1aaf939b1-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-2hcp9\" (UID: \"ded7be50-33c3-48ec-9c1e-cab1aaf939b1\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.377575 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.377518    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ded7be50-33c3-48ec-9c1e-cab1aaf939b1-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-2hcp9\" (UID: \"ded7be50-33c3-48ec-9c1e-cab1aaf939b1\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.377708 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.377673    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ded7be50-33c3-48ec-9c1e-cab1aaf939b1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-2hcp9\" (UID: \"ded7be50-33c3-48ec-9c1e-cab1aaf939b1\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.377750 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.377709    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd2zj\" (UniqueName: \"kubernetes.io/projected/ded7be50-33c3-48ec-9c1e-cab1aaf939b1-kube-api-access-hd2zj\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-2hcp9\" (UID: \"ded7be50-33c3-48ec-9c1e-cab1aaf939b1\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.387183 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.387159    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Nov 02 12:07:04.387340 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.387183    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 02 12:07:04.387340 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.387198    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-p8m6b\""
Nov 02 12:07:04.389371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.389352    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"]
Nov 02 12:07:04.478618 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.478576    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ded7be50-33c3-48ec-9c1e-cab1aaf939b1-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-2hcp9\" (UID: \"ded7be50-33c3-48ec-9c1e-cab1aaf939b1\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.478815 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.478626    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ded7be50-33c3-48ec-9c1e-cab1aaf939b1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-2hcp9\" (UID: \"ded7be50-33c3-48ec-9c1e-cab1aaf939b1\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.479087 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.479056    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ded7be50-33c3-48ec-9c1e-cab1aaf939b1-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-2hcp9\" (UID: \"ded7be50-33c3-48ec-9c1e-cab1aaf939b1\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.479228 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.479063    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ded7be50-33c3-48ec-9c1e-cab1aaf939b1-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-2hcp9\" (UID: \"ded7be50-33c3-48ec-9c1e-cab1aaf939b1\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.479228 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.479102    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ded7be50-33c3-48ec-9c1e-cab1aaf939b1-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-2hcp9\" (UID: \"ded7be50-33c3-48ec-9c1e-cab1aaf939b1\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.479228 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.479101    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ded7be50-33c3-48ec-9c1e-cab1aaf939b1-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-2hcp9\" (UID: \"ded7be50-33c3-48ec-9c1e-cab1aaf939b1\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.479228 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.479223    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ded7be50-33c3-48ec-9c1e-cab1aaf939b1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-2hcp9\" (UID: \"ded7be50-33c3-48ec-9c1e-cab1aaf939b1\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.479504 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.479252    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd2zj\" (UniqueName: \"kubernetes.io/projected/ded7be50-33c3-48ec-9c1e-cab1aaf939b1-kube-api-access-hd2zj\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-2hcp9\" (UID: \"ded7be50-33c3-48ec-9c1e-cab1aaf939b1\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.479504 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.479454    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ded7be50-33c3-48ec-9c1e-cab1aaf939b1-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-2hcp9\" (UID: \"ded7be50-33c3-48ec-9c1e-cab1aaf939b1\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.482160 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.482132    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ded7be50-33c3-48ec-9c1e-cab1aaf939b1-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-2hcp9\" (UID: \"ded7be50-33c3-48ec-9c1e-cab1aaf939b1\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.482387 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.482369    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ded7be50-33c3-48ec-9c1e-cab1aaf939b1-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-2hcp9\" (UID: \"ded7be50-33c3-48ec-9c1e-cab1aaf939b1\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.540099 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.540034    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd2zj\" (UniqueName: \"kubernetes.io/projected/ded7be50-33c3-48ec-9c1e-cab1aaf939b1-kube-api-access-hd2zj\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-2hcp9\" (UID: \"ded7be50-33c3-48ec-9c1e-cab1aaf939b1\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.598381 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.598346    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:04.981125 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.981096    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"]
Nov 02 12:07:04.988648 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:07:04.988622    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podded7be50_33c3_48ec_9c1e_cab1aaf939b1.slice/crio-e79f1fd3966eea3a8b7f39814b4afd732178d57609f2d26871aa56eedfa654c0 WatchSource:0}: Error finding container e79f1fd3966eea3a8b7f39814b4afd732178d57609f2d26871aa56eedfa654c0: Status 404 returned error can't find the container with id e79f1fd3966eea3a8b7f39814b4afd732178d57609f2d26871aa56eedfa654c0
Nov 02 12:07:04.989757 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:04.989730    4942 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 12:07:05.789062 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:05.788968    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9" event={"ID":"ded7be50-33c3-48ec-9c1e-cab1aaf939b1","Type":"ContainerStarted","Data":"e79f1fd3966eea3a8b7f39814b4afd732178d57609f2d26871aa56eedfa654c0"}
Nov 02 12:07:06.794984 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:06.794896    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9" event={"ID":"ded7be50-33c3-48ec-9c1e-cab1aaf939b1","Type":"ContainerStarted","Data":"8b1e0df20dabab3d8f504a23acc1faed188aad712159e53b8b8ff525ea25061b"}
Nov 02 12:07:06.913197 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:06.913147    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9" podStartSLOduration=1.405544613 podStartE2EDuration="2.913133208s" totalImagesPullingTime="1.507588595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:07:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:07:06.911221081 +0000 UTC m=+653.482570865" watchObservedRunningTime="2026-11-02 12:07:06.913133208 +0000 UTC m=+653.484483357"
Nov 02 12:07:07.598424 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.598396    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:07.694667 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.694635    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"]
Nov 02 12:07:07.698254 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.698230    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:07.774546 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.774513    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"]
Nov 02 12:07:07.800811 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.800778    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Nov 02 12:07:07.807610 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.807578    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebd62eee-d630-4bab-8d45-b766272f482a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q\" (UID: \"ebd62eee-d630-4bab-8d45-b766272f482a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:07.807743 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.807620    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebd62eee-d630-4bab-8d45-b766272f482a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q\" (UID: \"ebd62eee-d630-4bab-8d45-b766272f482a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:07.807743 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.807637    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebd62eee-d630-4bab-8d45-b766272f482a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q\" (UID: \"ebd62eee-d630-4bab-8d45-b766272f482a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:07.807851 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.807750    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebd62eee-d630-4bab-8d45-b766272f482a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q\" (UID: \"ebd62eee-d630-4bab-8d45-b766272f482a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:07.807851 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.807786    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7trn\" (UniqueName: \"kubernetes.io/projected/ebd62eee-d630-4bab-8d45-b766272f482a-kube-api-access-z7trn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q\" (UID: \"ebd62eee-d630-4bab-8d45-b766272f482a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:07.807851 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.807814    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebd62eee-d630-4bab-8d45-b766272f482a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q\" (UID: \"ebd62eee-d630-4bab-8d45-b766272f482a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:07.908955 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.908912    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebd62eee-d630-4bab-8d45-b766272f482a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q\" (UID: \"ebd62eee-d630-4bab-8d45-b766272f482a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:07.909163 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.909054    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebd62eee-d630-4bab-8d45-b766272f482a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q\" (UID: \"ebd62eee-d630-4bab-8d45-b766272f482a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:07.909163 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.909095    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebd62eee-d630-4bab-8d45-b766272f482a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q\" (UID: \"ebd62eee-d630-4bab-8d45-b766272f482a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:07.909163 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.909136    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebd62eee-d630-4bab-8d45-b766272f482a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q\" (UID: \"ebd62eee-d630-4bab-8d45-b766272f482a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:07.909395 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.909184    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7trn\" (UniqueName: \"kubernetes.io/projected/ebd62eee-d630-4bab-8d45-b766272f482a-kube-api-access-z7trn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q\" (UID: \"ebd62eee-d630-4bab-8d45-b766272f482a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:07.909395 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.909234    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebd62eee-d630-4bab-8d45-b766272f482a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q\" (UID: \"ebd62eee-d630-4bab-8d45-b766272f482a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:07.909395 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.909333    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ebd62eee-d630-4bab-8d45-b766272f482a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q\" (UID: \"ebd62eee-d630-4bab-8d45-b766272f482a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:07.909547 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.909517    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ebd62eee-d630-4bab-8d45-b766272f482a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q\" (UID: \"ebd62eee-d630-4bab-8d45-b766272f482a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:07.909642 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.909620    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ebd62eee-d630-4bab-8d45-b766272f482a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q\" (UID: \"ebd62eee-d630-4bab-8d45-b766272f482a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:07.911476 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.911452    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ebd62eee-d630-4bab-8d45-b766272f482a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q\" (UID: \"ebd62eee-d630-4bab-8d45-b766272f482a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:07.911946 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.911926    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebd62eee-d630-4bab-8d45-b766272f482a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q\" (UID: \"ebd62eee-d630-4bab-8d45-b766272f482a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:07.971980 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:07.971941    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7trn\" (UniqueName: \"kubernetes.io/projected/ebd62eee-d630-4bab-8d45-b766272f482a-kube-api-access-z7trn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q\" (UID: \"ebd62eee-d630-4bab-8d45-b766272f482a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:08.022846 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:08.022811    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:08.404439 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:08.401736    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"]
Nov 02 12:07:08.405826 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:07:08.405782    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebd62eee_d630_4bab_8d45_b766272f482a.slice/crio-184bf82bdc443399962fb8283576577e978c75702dea5734021fb982b791c6e5 WatchSource:0}: Error finding container 184bf82bdc443399962fb8283576577e978c75702dea5734021fb982b791c6e5: Status 404 returned error can't find the container with id 184bf82bdc443399962fb8283576577e978c75702dea5734021fb982b791c6e5
Nov 02 12:07:08.807577 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:08.807493    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q" event={"ID":"ebd62eee-d630-4bab-8d45-b766272f482a","Type":"ContainerStarted","Data":"184bf82bdc443399962fb8283576577e978c75702dea5734021fb982b791c6e5"}
Nov 02 12:07:09.813846 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:09.813805    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q" event={"ID":"ebd62eee-d630-4bab-8d45-b766272f482a","Type":"ContainerStarted","Data":"1e850760eddc13175742587ad38de9b602efc5138a4986628af6d9b73e23bf96"}
Nov 02 12:07:09.930960 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:09.930803    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q" podStartSLOduration=2.326467764 podStartE2EDuration="2.930787515s" totalImagesPullingTime="604.319751ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:07:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:07:09.930248523 +0000 UTC m=+656.501598280" watchObservedRunningTime="2026-11-02 12:07:09.930787515 +0000 UTC m=+656.502137270"
Nov 02 12:07:10.023040 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:10.023012    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:14.598881 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:14.598842    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:14.618286 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:14.618248    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:14.850967 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:14.850949    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-2hcp9"
Nov 02 12:07:15.830487 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.830445    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"]
Nov 02 12:07:15.834365 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.834341    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:15.884821 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.884789    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/430e2338-5daf-4020-9fc0-5645de2bf124-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh\" (UID: \"430e2338-5daf-4020-9fc0-5645de2bf124\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:15.884966 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.884891    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btzck\" (UniqueName: \"kubernetes.io/projected/430e2338-5daf-4020-9fc0-5645de2bf124-kube-api-access-btzck\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh\" (UID: \"430e2338-5daf-4020-9fc0-5645de2bf124\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:15.884966 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.884916    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/430e2338-5daf-4020-9fc0-5645de2bf124-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh\" (UID: \"430e2338-5daf-4020-9fc0-5645de2bf124\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:15.885048 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.885006    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/430e2338-5daf-4020-9fc0-5645de2bf124-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh\" (UID: \"430e2338-5daf-4020-9fc0-5645de2bf124\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:15.885048 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.885045    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/430e2338-5daf-4020-9fc0-5645de2bf124-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh\" (UID: \"430e2338-5daf-4020-9fc0-5645de2bf124\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:15.885126 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.885059    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/430e2338-5daf-4020-9fc0-5645de2bf124-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh\" (UID: \"430e2338-5daf-4020-9fc0-5645de2bf124\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:15.935822 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.935791    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Nov 02 12:07:15.938470 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.938439    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"]
Nov 02 12:07:15.986169 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.986134    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btzck\" (UniqueName: \"kubernetes.io/projected/430e2338-5daf-4020-9fc0-5645de2bf124-kube-api-access-btzck\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh\" (UID: \"430e2338-5daf-4020-9fc0-5645de2bf124\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:15.986169 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.986170    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/430e2338-5daf-4020-9fc0-5645de2bf124-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh\" (UID: \"430e2338-5daf-4020-9fc0-5645de2bf124\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:15.986436 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.986203    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/430e2338-5daf-4020-9fc0-5645de2bf124-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh\" (UID: \"430e2338-5daf-4020-9fc0-5645de2bf124\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:15.986436 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.986227    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/430e2338-5daf-4020-9fc0-5645de2bf124-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh\" (UID: \"430e2338-5daf-4020-9fc0-5645de2bf124\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:15.986436 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.986241    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/430e2338-5daf-4020-9fc0-5645de2bf124-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh\" (UID: \"430e2338-5daf-4020-9fc0-5645de2bf124\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:15.986436 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.986274    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/430e2338-5daf-4020-9fc0-5645de2bf124-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh\" (UID: \"430e2338-5daf-4020-9fc0-5645de2bf124\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:15.986641 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.986621    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/430e2338-5daf-4020-9fc0-5645de2bf124-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh\" (UID: \"430e2338-5daf-4020-9fc0-5645de2bf124\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:15.986641 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.986637    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/430e2338-5daf-4020-9fc0-5645de2bf124-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh\" (UID: \"430e2338-5daf-4020-9fc0-5645de2bf124\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:15.986739 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.986722    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/430e2338-5daf-4020-9fc0-5645de2bf124-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh\" (UID: \"430e2338-5daf-4020-9fc0-5645de2bf124\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:15.988549 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.988525    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/430e2338-5daf-4020-9fc0-5645de2bf124-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh\" (UID: \"430e2338-5daf-4020-9fc0-5645de2bf124\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:15.988791 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:15.988773    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/430e2338-5daf-4020-9fc0-5645de2bf124-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh\" (UID: \"430e2338-5daf-4020-9fc0-5645de2bf124\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:16.046790 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:16.046764    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btzck\" (UniqueName: \"kubernetes.io/projected/430e2338-5daf-4020-9fc0-5645de2bf124-kube-api-access-btzck\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh\" (UID: \"430e2338-5daf-4020-9fc0-5645de2bf124\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:16.147349 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:16.147323    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:16.526858 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:16.526832    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"]
Nov 02 12:07:16.528557 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:07:16.528527    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod430e2338_5daf_4020_9fc0_5645de2bf124.slice/crio-e33c677b941a48ab400a28f59a1c4c2c9a7d65d342caf1aa9dede30fcfba4493 WatchSource:0}: Error finding container e33c677b941a48ab400a28f59a1c4c2c9a7d65d342caf1aa9dede30fcfba4493: Status 404 returned error can't find the container with id e33c677b941a48ab400a28f59a1c4c2c9a7d65d342caf1aa9dede30fcfba4493
Nov 02 12:07:16.852428 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:16.852390    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh" event={"ID":"430e2338-5daf-4020-9fc0-5645de2bf124","Type":"ContainerStarted","Data":"e33c677b941a48ab400a28f59a1c4c2c9a7d65d342caf1aa9dede30fcfba4493"}
Nov 02 12:07:17.859693 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:17.859643    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh" event={"ID":"430e2338-5daf-4020-9fc0-5645de2bf124","Type":"ContainerStarted","Data":"2574d386ad49550e3ab4b96c8087579ba860e3c4a03a476481f33490874e5be9"}
Nov 02 12:07:17.978146 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:17.978091    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh" podStartSLOduration=2.411021177 podStartE2EDuration="2.978077714s" totalImagesPullingTime="567.056537ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:07:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:07:17.976535646 +0000 UTC m=+664.547885402" watchObservedRunningTime="2026-11-02 12:07:17.978077714 +0000 UTC m=+664.549427471"
Nov 02 12:07:18.023388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:18.023358    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:18.035765 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:18.035732    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:18.147773 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:18.147670    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:18.878760 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:18.878728    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q"
Nov 02 12:07:20.514292 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.514259    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"]
Nov 02 12:07:20.518580 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.518556    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.524854 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.524821    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/242aa9d0-49d5-4981-ad74-0f7b95635e6a-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw\" (UID: \"242aa9d0-49d5-4981-ad74-0f7b95635e6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.525102 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.525077    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/242aa9d0-49d5-4981-ad74-0f7b95635e6a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw\" (UID: \"242aa9d0-49d5-4981-ad74-0f7b95635e6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.525232 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.525124    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/242aa9d0-49d5-4981-ad74-0f7b95635e6a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw\" (UID: \"242aa9d0-49d5-4981-ad74-0f7b95635e6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.525232 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.525192    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w55d4\" (UniqueName: \"kubernetes.io/projected/242aa9d0-49d5-4981-ad74-0f7b95635e6a-kube-api-access-w55d4\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw\" (UID: \"242aa9d0-49d5-4981-ad74-0f7b95635e6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.525389 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.525341    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/242aa9d0-49d5-4981-ad74-0f7b95635e6a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw\" (UID: \"242aa9d0-49d5-4981-ad74-0f7b95635e6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.525442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.525404    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/242aa9d0-49d5-4981-ad74-0f7b95635e6a-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw\" (UID: \"242aa9d0-49d5-4981-ad74-0f7b95635e6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.622711 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.622680    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Nov 02 12:07:20.624583 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.624564    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"]
Nov 02 12:07:20.626013 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.625998    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/242aa9d0-49d5-4981-ad74-0f7b95635e6a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw\" (UID: \"242aa9d0-49d5-4981-ad74-0f7b95635e6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.626079 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.626025    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/242aa9d0-49d5-4981-ad74-0f7b95635e6a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw\" (UID: \"242aa9d0-49d5-4981-ad74-0f7b95635e6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.626079 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.626051    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w55d4\" (UniqueName: \"kubernetes.io/projected/242aa9d0-49d5-4981-ad74-0f7b95635e6a-kube-api-access-w55d4\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw\" (UID: \"242aa9d0-49d5-4981-ad74-0f7b95635e6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.626079 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.626073    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/242aa9d0-49d5-4981-ad74-0f7b95635e6a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw\" (UID: \"242aa9d0-49d5-4981-ad74-0f7b95635e6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.626211 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.626192    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/242aa9d0-49d5-4981-ad74-0f7b95635e6a-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw\" (UID: \"242aa9d0-49d5-4981-ad74-0f7b95635e6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.626285 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.626272    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/242aa9d0-49d5-4981-ad74-0f7b95635e6a-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw\" (UID: \"242aa9d0-49d5-4981-ad74-0f7b95635e6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.626686 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.626516    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/242aa9d0-49d5-4981-ad74-0f7b95635e6a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw\" (UID: \"242aa9d0-49d5-4981-ad74-0f7b95635e6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.626686 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.626546    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/242aa9d0-49d5-4981-ad74-0f7b95635e6a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw\" (UID: \"242aa9d0-49d5-4981-ad74-0f7b95635e6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.626686 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.626628    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/242aa9d0-49d5-4981-ad74-0f7b95635e6a-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw\" (UID: \"242aa9d0-49d5-4981-ad74-0f7b95635e6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.628434 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.628408    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/242aa9d0-49d5-4981-ad74-0f7b95635e6a-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw\" (UID: \"242aa9d0-49d5-4981-ad74-0f7b95635e6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.628536 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.628523    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/242aa9d0-49d5-4981-ad74-0f7b95635e6a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw\" (UID: \"242aa9d0-49d5-4981-ad74-0f7b95635e6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.688013 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.687974    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w55d4\" (UniqueName: \"kubernetes.io/projected/242aa9d0-49d5-4981-ad74-0f7b95635e6a-kube-api-access-w55d4\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw\" (UID: \"242aa9d0-49d5-4981-ad74-0f7b95635e6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:20.830799 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:20.830711    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:21.232471 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:21.232438    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"]
Nov 02 12:07:21.234116 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:07:21.234089    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod242aa9d0_49d5_4981_ad74_0f7b95635e6a.slice/crio-8c308c76ecba6a384534181cbadf442ca8f9488aacebd0e22ee0dce8c3da643a WatchSource:0}: Error finding container 8c308c76ecba6a384534181cbadf442ca8f9488aacebd0e22ee0dce8c3da643a: Status 404 returned error can't find the container with id 8c308c76ecba6a384534181cbadf442ca8f9488aacebd0e22ee0dce8c3da643a
Nov 02 12:07:21.921725 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:21.921688    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw" event={"ID":"242aa9d0-49d5-4981-ad74-0f7b95635e6a","Type":"ContainerStarted","Data":"4c5c41c79c314fba28ca9a5e00d2fef079b74d2834606a94202cd5d0c5173d04"}
Nov 02 12:07:21.922064 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:21.921742    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw" event={"ID":"242aa9d0-49d5-4981-ad74-0f7b95635e6a","Type":"ContainerStarted","Data":"8c308c76ecba6a384534181cbadf442ca8f9488aacebd0e22ee0dce8c3da643a"}
Nov 02 12:07:22.038943 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:22.038892    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw" podStartSLOduration=1.450426809 podStartE2EDuration="2.038879459s" totalImagesPullingTime="588.45265ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:07:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:07:22.03822598 +0000 UTC m=+668.609575737" watchObservedRunningTime="2026-11-02 12:07:22.038879459 +0000 UTC m=+668.610229216"
Nov 02 12:07:22.831110 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:22.831070    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:26.147736 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:26.147658    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:26.163776 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:26.163748    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:26.957349 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:26.957314    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh"
Nov 02 12:07:28.565308 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.565269    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"]
Nov 02 12:07:28.569576 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.569551    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.607242 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.607205    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.607406 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.607241    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgg67\" (UniqueName: \"kubernetes.io/projected/72e2083e-9c67-4b7b-b583-7a1259ee4705-kube-api-access-qgg67\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.607406 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.607341    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.607406 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.607366    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.607406 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.607385    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.607641 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.607418    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72e2083e-9c67-4b7b-b583-7a1259ee4705-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.607641 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.607578    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.607641 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.607608    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.607641 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.607635    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.675012 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.674963    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"]
Nov 02 12:07:28.710800 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.710750    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.710800 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.710800    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgg67\" (UniqueName: \"kubernetes.io/projected/72e2083e-9c67-4b7b-b583-7a1259ee4705-kube-api-access-qgg67\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.711044 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.710844    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.711044 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.710878    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.711044 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.710903    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.711044 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.710950    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72e2083e-9c67-4b7b-b583-7a1259ee4705-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.711044 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.711010    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.711044 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.711035    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.711430 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.711069    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.713636 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.713603    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.713768 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.713747    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.713897 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.713869    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.714382 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.714358    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72e2083e-9c67-4b7b-b583-7a1259ee4705-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.714502 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.714483    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.715105 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.715081    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.717088 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.717061    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.773363 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.773335    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.773649 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.773624    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgg67\" (UniqueName: \"kubernetes.io/projected/72e2083e-9c67-4b7b-b583-7a1259ee4705-kube-api-access-qgg67\") pod \"maas-default-gateway-openshift-default-6dcf757899-zkvl6\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:28.887034 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:28.886977    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:29.374398 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:29.374363    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"]
Nov 02 12:07:29.375606 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:07:29.375573    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e2083e_9c67_4b7b_b583_7a1259ee4705.slice/crio-c268c576655862afb363acf4f03ae669f0ba6d972362a44eed725a700f034d9a WatchSource:0}: Error finding container c268c576655862afb363acf4f03ae669f0ba6d972362a44eed725a700f034d9a: Status 404 returned error can't find the container with id c268c576655862afb363acf4f03ae669f0ba6d972362a44eed725a700f034d9a
Nov 02 12:07:29.378718 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:29.378681    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:07:29.378802 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:29.378773    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:07:29.378873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:29.378853    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:07:29.977862 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:29.977822    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6" event={"ID":"72e2083e-9c67-4b7b-b583-7a1259ee4705","Type":"ContainerStarted","Data":"05ea3f519b19b7de9c67354a4affecdbbdfc77719b20bea828b0b9aac897a4c6"}
Nov 02 12:07:29.978513 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:29.978486    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6" event={"ID":"72e2083e-9c67-4b7b-b583-7a1259ee4705","Type":"ContainerStarted","Data":"c268c576655862afb363acf4f03ae669f0ba6d972362a44eed725a700f034d9a"}
Nov 02 12:07:30.149052 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:30.148984    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6" podStartSLOduration=2.148965306 podStartE2EDuration="2.148965306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:07:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:07:30.148501878 +0000 UTC m=+676.719851658" watchObservedRunningTime="2026-11-02 12:07:30.148965306 +0000 UTC m=+676.720315064"
Nov 02 12:07:30.831408 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:30.831367    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:30.850101 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:30.850076    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:30.888126 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:30.888078    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:30.889293 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:30.889266    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-zkvl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 02 12:07:30.889447 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:30.889336    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6" podUID="72e2083e-9c67-4b7b-b583-7a1259ee4705" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 02 12:07:31.005728 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:31.005699    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:07:31.010549 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:07:31.010515    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:07:31.887548 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:31.887514    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-zkvl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 02 12:07:31.887722 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:31.887575    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6" podUID="72e2083e-9c67-4b7b-b583-7a1259ee4705" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 02 12:07:32.887536 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:32.887504    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-zkvl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 02 12:07:32.888003 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:32.887560    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6" podUID="72e2083e-9c67-4b7b-b583-7a1259ee4705" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 02 12:07:33.888023 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:33.887990    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-zkvl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 02 12:07:33.888428 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:33.888051    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6" podUID="72e2083e-9c67-4b7b-b583-7a1259ee4705" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 02 12:07:34.888182 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:34.888147    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-zkvl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 02 12:07:34.888691 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:34.888203    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6" podUID="72e2083e-9c67-4b7b-b583-7a1259ee4705" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 02 12:07:35.888071 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:35.888033    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-zkvl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 02 12:07:35.888242 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:35.888108    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6" podUID="72e2083e-9c67-4b7b-b583-7a1259ee4705" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 02 12:07:36.887933 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:36.887894    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-zkvl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Nov 02 12:07:36.888107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:36.887960    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6" podUID="72e2083e-9c67-4b7b-b583-7a1259ee4705" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Nov 02 12:07:38.586029 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:38.586000    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:38.586504 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:38.586275    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:38.587127 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:38.587106    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:07:43.692148 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.692113    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"]
Nov 02 12:07:43.698602 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.698583    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.798833 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.798807    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"]
Nov 02 12:07:43.875287 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.875252    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.875457 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.875372    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8flh\" (UniqueName: \"kubernetes.io/projected/73d2d79d-46c2-44c2-ae4a-3211e52244f4-kube-api-access-g8flh\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.875512 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.875463    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.875580 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.875563    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.875627 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.875617    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.875703 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.875688    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.875766 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.875751    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.875803 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.875789    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.875844 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.875817    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.976635 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.976562    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.976635 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.976610    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.976853 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.976640    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.976853 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.976665    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.976853 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.976685    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.976853 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.976713    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.976853 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.976747    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8flh\" (UniqueName: \"kubernetes.io/projected/73d2d79d-46c2-44c2-ae4a-3211e52244f4-kube-api-access-g8flh\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.976853 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.976814    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.976853 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.976847    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.977249 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.977010    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.977249 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.977114    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.977249 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.977146    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.977467 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.977278    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.977649 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.977628    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.979050 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.979031    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:43.979107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:43.979092    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:44.037349 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:44.037325    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:44.037462 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:44.037446    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8flh\" (UniqueName: \"kubernetes.io/projected/73d2d79d-46c2-44c2-ae4a-3211e52244f4-kube-api-access-g8flh\") pod \"maas-default-gateway-openshift-default-6dcf757899-sn8dd\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:44.313315 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:44.313217    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:44.741087 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:44.741063    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"]
Nov 02 12:07:44.742591 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:07:44.742559    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73d2d79d_46c2_44c2_ae4a_3211e52244f4.slice/crio-40f135d4e191419c50303f6c306ae728179958c1c1ac4140336b1450a525b93a WatchSource:0}: Error finding container 40f135d4e191419c50303f6c306ae728179958c1c1ac4140336b1450a525b93a: Status 404 returned error can't find the container with id 40f135d4e191419c50303f6c306ae728179958c1c1ac4140336b1450a525b93a
Nov 02 12:07:44.744827 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:44.744794    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:07:44.744909 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:44.744862    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:07:44.744952 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:44.744926    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:07:45.067909 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:45.067827    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd" event={"ID":"73d2d79d-46c2-44c2-ae4a-3211e52244f4","Type":"ContainerStarted","Data":"e757a6685f93072f361a2b3e88cd6b0c17dfba245756f2a3714f5bfc7e0fef05"}
Nov 02 12:07:45.068063 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:45.067911    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd" event={"ID":"73d2d79d-46c2-44c2-ae4a-3211e52244f4","Type":"ContainerStarted","Data":"40f135d4e191419c50303f6c306ae728179958c1c1ac4140336b1450a525b93a"}
Nov 02 12:07:45.194324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:45.193438    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd" podStartSLOduration=2.193418177 podStartE2EDuration="2.193418177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:07:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:07:45.191940497 +0000 UTC m=+691.763290256" watchObservedRunningTime="2026-11-02 12:07:45.193418177 +0000 UTC m=+691.764767920"
Nov 02 12:07:45.313897 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:45.313858    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:45.315128 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:45.315100    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-sn8dd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 02 12:07:45.315256 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:45.315151    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd" podUID="73d2d79d-46c2-44c2-ae4a-3211e52244f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 02 12:07:46.314439 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:46.314405    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-sn8dd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 02 12:07:46.314934 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:46.314467    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd" podUID="73d2d79d-46c2-44c2-ae4a-3211e52244f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 02 12:07:47.314269 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:47.314235    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-sn8dd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 02 12:07:47.314464 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:47.314292    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd" podUID="73d2d79d-46c2-44c2-ae4a-3211e52244f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 02 12:07:48.314465 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:48.314427    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-sn8dd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 02 12:07:48.314893 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:48.314483    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd" podUID="73d2d79d-46c2-44c2-ae4a-3211e52244f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 02 12:07:49.313711 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:49.313679    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-sn8dd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 02 12:07:49.313901 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:49.313747    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd" podUID="73d2d79d-46c2-44c2-ae4a-3211e52244f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 02 12:07:50.314580 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:50.314541    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-sn8dd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 02 12:07:50.315056 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:50.314606    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd" podUID="73d2d79d-46c2-44c2-ae4a-3211e52244f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 02 12:07:50.980906 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:50.980860    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-fb96bcc76-d9wdp"]
Nov 02 12:07:50.984854 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:50.984828    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fb96bcc76-d9wdp"
Nov 02 12:07:51.041584 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:51.041555    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-fb96bcc76-d9wdp"]
Nov 02 12:07:51.047698 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:51.047667    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lznd\" (UniqueName: \"kubernetes.io/projected/a8304b1f-5415-4992-a8d5-23e2c27e6b28-kube-api-access-5lznd\") pod \"authorino-fb96bcc76-d9wdp\" (UID: \"a8304b1f-5415-4992-a8d5-23e2c27e6b28\") " pod="kuadrant-system/authorino-fb96bcc76-d9wdp"
Nov 02 12:07:51.047843 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:51.047796    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a8304b1f-5415-4992-a8d5-23e2c27e6b28-tls-cert\") pod \"authorino-fb96bcc76-d9wdp\" (UID: \"a8304b1f-5415-4992-a8d5-23e2c27e6b28\") " pod="kuadrant-system/authorino-fb96bcc76-d9wdp"
Nov 02 12:07:51.149200 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:51.149161    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lznd\" (UniqueName: \"kubernetes.io/projected/a8304b1f-5415-4992-a8d5-23e2c27e6b28-kube-api-access-5lznd\") pod \"authorino-fb96bcc76-d9wdp\" (UID: \"a8304b1f-5415-4992-a8d5-23e2c27e6b28\") " pod="kuadrant-system/authorino-fb96bcc76-d9wdp"
Nov 02 12:07:51.149410 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:51.149220    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a8304b1f-5415-4992-a8d5-23e2c27e6b28-tls-cert\") pod \"authorino-fb96bcc76-d9wdp\" (UID: \"a8304b1f-5415-4992-a8d5-23e2c27e6b28\") " pod="kuadrant-system/authorino-fb96bcc76-d9wdp"
Nov 02 12:07:51.151595 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:51.151569    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a8304b1f-5415-4992-a8d5-23e2c27e6b28-tls-cert\") pod \"authorino-fb96bcc76-d9wdp\" (UID: \"a8304b1f-5415-4992-a8d5-23e2c27e6b28\") " pod="kuadrant-system/authorino-fb96bcc76-d9wdp"
Nov 02 12:07:51.210201 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:51.210169    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lznd\" (UniqueName: \"kubernetes.io/projected/a8304b1f-5415-4992-a8d5-23e2c27e6b28-kube-api-access-5lznd\") pod \"authorino-fb96bcc76-d9wdp\" (UID: \"a8304b1f-5415-4992-a8d5-23e2c27e6b28\") " pod="kuadrant-system/authorino-fb96bcc76-d9wdp"
Nov 02 12:07:51.298260 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:51.298193    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-fb96bcc76-d9wdp"
Nov 02 12:07:51.313697 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:51.313663    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-sn8dd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 02 12:07:51.313809 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:51.313730    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd" podUID="73d2d79d-46c2-44c2-ae4a-3211e52244f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 02 12:07:51.686191 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:51.686165    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-fb96bcc76-d9wdp"]
Nov 02 12:07:51.688008 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:07:51.687976    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8304b1f_5415_4992_a8d5_23e2c27e6b28.slice/crio-8940a3b6ad491e0b27831eabb23c14b378c0a2fb6d000734a1ebdcf0b5375ec7 WatchSource:0}: Error finding container 8940a3b6ad491e0b27831eabb23c14b378c0a2fb6d000734a1ebdcf0b5375ec7: Status 404 returned error can't find the container with id 8940a3b6ad491e0b27831eabb23c14b378c0a2fb6d000734a1ebdcf0b5375ec7
Nov 02 12:07:52.114607 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:52.114572    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fb96bcc76-d9wdp" event={"ID":"a8304b1f-5415-4992-a8d5-23e2c27e6b28","Type":"ContainerStarted","Data":"8940a3b6ad491e0b27831eabb23c14b378c0a2fb6d000734a1ebdcf0b5375ec7"}
Nov 02 12:07:52.313958 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:52.313917    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-sn8dd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Nov 02 12:07:52.314172 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:52.313972    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd" podUID="73d2d79d-46c2-44c2-ae4a-3211e52244f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Nov 02 12:07:54.017783 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:54.017742    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:54.018221 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:54.018185    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:54.018968 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:54.018949    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:07:54.126419 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:54.126382    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-fb96bcc76-d9wdp" event={"ID":"a8304b1f-5415-4992-a8d5-23e2c27e6b28","Type":"ContainerStarted","Data":"203ca25761f87f4a65898530a5cc85d38049bfcafdba9107782de5f25b868eb3"}
Nov 02 12:07:54.458806 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:54.458744    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-fb96bcc76-d9wdp" podStartSLOduration=2.703655541 podStartE2EDuration="4.458731051s" totalImagesPullingTime="1.75507551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:07:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:07:54.458544973 +0000 UTC m=+701.029894723" watchObservedRunningTime="2026-11-02 12:07:54.458731051 +0000 UTC m=+701.030080808"
Nov 02 12:07:54.496859 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:54.496819    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-bc9d986f5-bqcdf"]
Nov 02 12:07:54.497089 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:54.497062    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-bc9d986f5-bqcdf" podUID="8ea0a699-2c5b-46f0-8141-bb9a1396d76d" containerName="authorino" containerID="cri-o://f09a0115211083d18aea72109e33ea0dcdec8a1d6fa31d0244f7b5ec2baf373b" gracePeriod=30
Nov 02 12:07:54.746729 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:54.746696    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bc9d986f5-bqcdf"
Nov 02 12:07:54.795809 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:54.795768    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ea0a699-2c5b-46f0-8141-bb9a1396d76d-kube-api-access-ztdwv\" (UniqueName: \"kubernetes.io/projected/8ea0a699-2c5b-46f0-8141-bb9a1396d76d-kube-api-access-ztdwv\") pod \"8ea0a699-2c5b-46f0-8141-bb9a1396d76d\" (UID: \"8ea0a699-2c5b-46f0-8141-bb9a1396d76d\") "
Nov 02 12:07:54.796010 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:54.795873    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8ea0a699-2c5b-46f0-8141-bb9a1396d76d-tls-cert\" (UniqueName: \"kubernetes.io/secret/8ea0a699-2c5b-46f0-8141-bb9a1396d76d-tls-cert\") pod \"8ea0a699-2c5b-46f0-8141-bb9a1396d76d\" (UID: \"8ea0a699-2c5b-46f0-8141-bb9a1396d76d\") "
Nov 02 12:07:54.806327 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:54.802421    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ea0a699-2c5b-46f0-8141-bb9a1396d76d-kube-api-access-ztdwv" pod "8ea0a699-2c5b-46f0-8141-bb9a1396d76d" (UID: "8ea0a699-2c5b-46f0-8141-bb9a1396d76d"). InnerVolumeSpecName "kube-api-access-ztdwv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:07:54.826569 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:54.826531    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ea0a699-2c5b-46f0-8141-bb9a1396d76d-tls-cert" pod "8ea0a699-2c5b-46f0-8141-bb9a1396d76d" (UID: "8ea0a699-2c5b-46f0-8141-bb9a1396d76d"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:07:54.896910 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:54.896866    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztdwv\" (UniqueName: \"kubernetes.io/projected/8ea0a699-2c5b-46f0-8141-bb9a1396d76d-kube-api-access-ztdwv\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:07:54.896910 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:54.896912    4942 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8ea0a699-2c5b-46f0-8141-bb9a1396d76d-tls-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:07:55.132906 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:55.132864    4942 generic.go:356] "Generic (PLEG): container finished" podID="8ea0a699-2c5b-46f0-8141-bb9a1396d76d" containerID="f09a0115211083d18aea72109e33ea0dcdec8a1d6fa31d0244f7b5ec2baf373b" exitCode=0
Nov 02 12:07:55.133411 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:55.132916    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bc9d986f5-bqcdf"
Nov 02 12:07:55.133411 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:55.132946    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bc9d986f5-bqcdf" event={"ID":"8ea0a699-2c5b-46f0-8141-bb9a1396d76d","Type":"ContainerDied","Data":"f09a0115211083d18aea72109e33ea0dcdec8a1d6fa31d0244f7b5ec2baf373b"}
Nov 02 12:07:55.133411 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:55.132995    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bc9d986f5-bqcdf" event={"ID":"8ea0a699-2c5b-46f0-8141-bb9a1396d76d","Type":"ContainerDied","Data":"375a28539d97ba7ef220c192055ec41c6272d5c6090e5455c7741868da2d9168"}
Nov 02 12:07:55.133411 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:55.133020    4942 scope.go:122] "RemoveContainer" containerID="f09a0115211083d18aea72109e33ea0dcdec8a1d6fa31d0244f7b5ec2baf373b"
Nov 02 12:07:55.142396 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:55.142379    4942 scope.go:122] "RemoveContainer" containerID="f09a0115211083d18aea72109e33ea0dcdec8a1d6fa31d0244f7b5ec2baf373b"
Nov 02 12:07:55.142692 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:07:55.142674    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f09a0115211083d18aea72109e33ea0dcdec8a1d6fa31d0244f7b5ec2baf373b\": container with ID starting with f09a0115211083d18aea72109e33ea0dcdec8a1d6fa31d0244f7b5ec2baf373b not found: ID does not exist" containerID="f09a0115211083d18aea72109e33ea0dcdec8a1d6fa31d0244f7b5ec2baf373b"
Nov 02 12:07:55.142742 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:55.142701    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f09a0115211083d18aea72109e33ea0dcdec8a1d6fa31d0244f7b5ec2baf373b"} err="failed to get container status \"f09a0115211083d18aea72109e33ea0dcdec8a1d6fa31d0244f7b5ec2baf373b\": rpc error: code = NotFound desc = could not find container \"f09a0115211083d18aea72109e33ea0dcdec8a1d6fa31d0244f7b5ec2baf373b\": container with ID starting with f09a0115211083d18aea72109e33ea0dcdec8a1d6fa31d0244f7b5ec2baf373b not found: ID does not exist"
Nov 02 12:07:55.296886 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:55.296843    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-bc9d986f5-bqcdf"]
Nov 02 12:07:55.301380 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:55.301358    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-bc9d986f5-bqcdf"]
Nov 02 12:07:55.543309 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:07:55.543265    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8ea0a699-2c5b-46f0-8141-bb9a1396d76d" path="/var/lib/kubelet/pods/8ea0a699-2c5b-46f0-8141-bb9a1396d76d/volumes"
Nov 02 12:08:00.982324 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:08:00.982270    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:08:26.452133 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.452084    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"]
Nov 02 12:08:26.453197 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.453166    4942 cpu_manager.go:414] "containerMap: removing container" podUID="8ea0a699-2c5b-46f0-8141-bb9a1396d76d" containerName="authorino"
Nov 02 12:08:26.453197 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.453198    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ea0a699-2c5b-46f0-8141-bb9a1396d76d" containerName="authorino"
Nov 02 12:08:26.453480 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.453462    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ea0a699-2c5b-46f0-8141-bb9a1396d76d" containerName="authorino"
Nov 02 12:08:26.463528 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.463502    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.517802 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.517765    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"]
Nov 02 12:08:26.563958 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.563931    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-b1703c-openshift-default-dockercfg-rb2rb\""
Nov 02 12:08:26.593137 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.593100    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-data\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.593369 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.593155    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-credential-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.593369 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.593195    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-workload-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.593369 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.593238    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-workload-certs\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.593369 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.593283    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-envoy\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.593697 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.593395    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-token\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.593697 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.593439    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e181569-4b01-4afe-9d02-278ff66af4ac-istiod-ca-cert\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.593697 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.593494    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhvcl\" (UniqueName: \"kubernetes.io/projected/1e181569-4b01-4afe-9d02-278ff66af4ac-kube-api-access-fhvcl\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.593697 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.593530    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-podinfo\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.694479 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.694448    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e181569-4b01-4afe-9d02-278ff66af4ac-istiod-ca-cert\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.694666 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.694502    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhvcl\" (UniqueName: \"kubernetes.io/projected/1e181569-4b01-4afe-9d02-278ff66af4ac-kube-api-access-fhvcl\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.694752 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.694710    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-podinfo\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.694819 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.694767    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-data\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.694819 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.694805    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-credential-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.694935 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.694849    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-workload-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.694935 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.694911    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-workload-certs\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.695028 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.694956    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-envoy\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.695087 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.695065    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-token\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.695164 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.695140    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-data\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.695226 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.695205    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-credential-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.695284 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.695231    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-workload-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.695479 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.695449    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e181569-4b01-4afe-9d02-278ff66af4ac-istiod-ca-cert\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.695584 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.695537    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-workload-certs\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.697204 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.697184    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-envoy\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.697371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.697352    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-podinfo\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.765545 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.765474    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-token\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.765682 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.765662    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhvcl\" (UniqueName: \"kubernetes.io/projected/1e181569-4b01-4afe-9d02-278ff66af4ac-kube-api-access-fhvcl\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:26.789321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:26.788391    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:27.219675 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:27.219648    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"]
Nov 02 12:08:27.222412 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:08:27.222381    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e181569_4b01_4afe_9d02_278ff66af4ac.slice/crio-904f9b6f2ec60b446cfb306b0061cdd200e3f029df648ae7c2892a6459e331a2 WatchSource:0}: Error finding container 904f9b6f2ec60b446cfb306b0061cdd200e3f029df648ae7c2892a6459e331a2: Status 404 returned error can't find the container with id 904f9b6f2ec60b446cfb306b0061cdd200e3f029df648ae7c2892a6459e331a2
Nov 02 12:08:27.224505 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:27.224469    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:08:27.224607 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:27.224543    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:08:27.224607 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:27.224573    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:08:27.307975 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:27.307936    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s" event={"ID":"1e181569-4b01-4afe-9d02-278ff66af4ac","Type":"ContainerStarted","Data":"a25837e3d41728c482673b3118b65c00e2491104ce4b2b7d12e279887b5c47a4"}
Nov 02 12:08:27.308168 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:27.307997    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s" event={"ID":"1e181569-4b01-4afe-9d02-278ff66af4ac","Type":"ContainerStarted","Data":"904f9b6f2ec60b446cfb306b0061cdd200e3f029df648ae7c2892a6459e331a2"}
Nov 02 12:08:27.447293 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:27.447232    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s" podStartSLOduration=1.4472130939999999 podStartE2EDuration="1.447213094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:08:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:08:27.445035173 +0000 UTC m=+734.016384930" watchObservedRunningTime="2026-11-02 12:08:27.447213094 +0000 UTC m=+734.018562853"
Nov 02 12:08:27.789043 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:27.789002    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:28.793783 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:28.793752    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:29.320735 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:29.320706    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:29.321509 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:29.321484    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:08:30.984072 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:08:30.984029    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:08:41.858052 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:41.858006    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"]
Nov 02 12:08:41.863137 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:41.863113    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:08:41.915291 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:41.915256    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"]
Nov 02 12:08:41.955463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:41.955432    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-maas-api-tls\") pod \"maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg\" (UID: \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:08:41.955609 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:41.955468    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg\" (UID: \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:08:41.955609 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:41.955555    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxp5s\" (UniqueName: \"kubernetes.io/projected/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-kube-api-access-mxp5s\") pod \"maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg\" (UID: \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:08:41.955609 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:41.955591    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-metrics-certs\") pod \"maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg\" (UID: \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:08:41.961279 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:41.961255    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 02 12:08:41.961419 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:41.961317    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-b6f3d8-serving-cert\""
Nov 02 12:08:41.961419 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:41.961352    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pl94d\""
Nov 02 12:08:42.056172 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.056137    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-maas-api-tls\") pod \"maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg\" (UID: \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:08:42.056355 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.056182    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg\" (UID: \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:08:42.056355 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.056225    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxp5s\" (UniqueName: \"kubernetes.io/projected/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-kube-api-access-mxp5s\") pod \"maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg\" (UID: \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:08:42.056355 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.056251    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-metrics-certs\") pod \"maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg\" (UID: \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:08:42.056355 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:08:42.056276    4942 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-serving-cert: secret "maas-api-e2e-praxis-a-b6f3d8-serving-cert" not found
Nov 02 12:08:42.056537 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:08:42.056383    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-maas-api-tls podName:3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b nodeName:}" failed. No retries permitted until 2026-11-02 12:08:42.556362374 +0000 UTC m=+749.127712131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-maas-api-tls") pod "maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg" (UID: "3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b") : secret "maas-api-e2e-praxis-a-b6f3d8-serving-cert" not found
Nov 02 12:08:42.056979 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.056962    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg\" (UID: \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:08:42.058710 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.058687    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-metrics-certs\") pod \"maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg\" (UID: \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:08:42.119190 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.119127    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxp5s\" (UniqueName: \"kubernetes.io/projected/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-kube-api-access-mxp5s\") pod \"maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg\" (UID: \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:08:42.314549 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.314516    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn"]
Nov 02 12:08:42.318969 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.318947    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn"
Nov 02 12:08:42.359375 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.359347    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9rw6\" (UniqueName: \"kubernetes.io/projected/cb8c82c6-339a-4880-a890-2b3effe841ae-kube-api-access-c9rw6\") pod \"payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn\" (UID: \"cb8c82c6-339a-4880-a890-2b3effe841ae\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn"
Nov 02 12:08:42.359493 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.359478    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb8c82c6-339a-4880-a890-2b3effe841ae-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn\" (UID: \"cb8c82c6-339a-4880-a890-2b3effe841ae\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn"
Nov 02 12:08:42.371292 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.371155    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-b6f3d8\""
Nov 02 12:08:42.371786 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.371762    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-b6f3d8-dockercfg-7j85g\""
Nov 02 12:08:42.374856 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.374833    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn"]
Nov 02 12:08:42.460636 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.460604    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9rw6\" (UniqueName: \"kubernetes.io/projected/cb8c82c6-339a-4880-a890-2b3effe841ae-kube-api-access-c9rw6\") pod \"payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn\" (UID: \"cb8c82c6-339a-4880-a890-2b3effe841ae\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn"
Nov 02 12:08:42.460778 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.460719    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb8c82c6-339a-4880-a890-2b3effe841ae-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn\" (UID: \"cb8c82c6-339a-4880-a890-2b3effe841ae\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn"
Nov 02 12:08:42.461331 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.461312    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb8c82c6-339a-4880-a890-2b3effe841ae-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn\" (UID: \"cb8c82c6-339a-4880-a890-2b3effe841ae\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn"
Nov 02 12:08:42.521068 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.521039    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9rw6\" (UniqueName: \"kubernetes.io/projected/cb8c82c6-339a-4880-a890-2b3effe841ae-kube-api-access-c9rw6\") pod \"payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn\" (UID: \"cb8c82c6-339a-4880-a890-2b3effe841ae\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn"
Nov 02 12:08:42.561738 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.561714    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-maas-api-tls\") pod \"maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg\" (UID: \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:08:42.563986 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.563961    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-maas-api-tls\") pod \"maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg\" (UID: \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:08:42.633543 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.633483    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn"
Nov 02 12:08:42.778786 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:42.778752    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:08:43.004815 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:43.004787    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn"]
Nov 02 12:08:43.006533 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:08:43.006506    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb8c82c6_339a_4880_a890_2b3effe841ae.slice/crio-c70a6dd8c0933e7ae9776fbb1541f14d566a3249582a735e56aed5e47246cfb0 WatchSource:0}: Error finding container c70a6dd8c0933e7ae9776fbb1541f14d566a3249582a735e56aed5e47246cfb0: Status 404 returned error can't find the container with id c70a6dd8c0933e7ae9776fbb1541f14d566a3249582a735e56aed5e47246cfb0
Nov 02 12:08:43.153600 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:43.153573    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"]
Nov 02 12:08:43.154473 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:08:43.154447    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ff3fe3d_cdc9_49ca_9ef6_89994a145a0b.slice/crio-3efce5e5682590de5d9081c61625a25a7017facd356efd584a7ddff45d5505a9 WatchSource:0}: Error finding container 3efce5e5682590de5d9081c61625a25a7017facd356efd584a7ddff45d5505a9: Status 404 returned error can't find the container with id 3efce5e5682590de5d9081c61625a25a7017facd356efd584a7ddff45d5505a9
Nov 02 12:08:43.400307 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:43.400263    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg" event={"ID":"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b","Type":"ContainerStarted","Data":"3efce5e5682590de5d9081c61625a25a7017facd356efd584a7ddff45d5505a9"}
Nov 02 12:08:43.401608 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:43.401580    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn" event={"ID":"cb8c82c6-339a-4880-a890-2b3effe841ae","Type":"ContainerStarted","Data":"c70a6dd8c0933e7ae9776fbb1541f14d566a3249582a735e56aed5e47246cfb0"}
Nov 02 12:08:46.424868 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:46.424798    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn" event={"ID":"cb8c82c6-339a-4880-a890-2b3effe841ae","Type":"ContainerStarted","Data":"452d921257b2193dd2757b1dd10e991fa5b984ca2500660078cc6a91e44f4835"}
Nov 02 12:08:46.425286 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:46.425069    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn"
Nov 02 12:08:46.541552 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:46.541495    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn" podStartSLOduration=1.21218038 podStartE2EDuration="4.541481733s" totalImagesPullingTime="3.329301353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:08:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:08:46.541097837 +0000 UTC m=+753.112447595" watchObservedRunningTime="2026-11-02 12:08:46.541481733 +0000 UTC m=+753.112831490"
Nov 02 12:08:47.431913 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:47.431870    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg" event={"ID":"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b","Type":"ContainerStarted","Data":"91b4658e66a7722ac9f12144f4f71be92234a7bddf4ba171b5fda5f1db17d853"}
Nov 02 12:08:47.549729 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:47.549678    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg" podStartSLOduration=3.002399516 podStartE2EDuration="6.549665016s" totalImagesPullingTime="3.5472655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:08:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:08:47.549127172 +0000 UTC m=+754.120476930" watchObservedRunningTime="2026-11-02 12:08:47.549665016 +0000 UTC m=+754.121014773"
Nov 02 12:08:48.437363 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:48.437335    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:08:54.449585 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:54.449509    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:08:57.435423 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:08:57.435394    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn"
Nov 02 12:09:00.985082 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:09:00.985043    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:09:03.538768 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:03.538730    4942 generic.go:356] "Generic (PLEG): container finished" podID="1e181569-4b01-4afe-9d02-278ff66af4ac" containerID="a25837e3d41728c482673b3118b65c00e2491104ce4b2b7d12e279887b5c47a4" exitCode=0
Nov 02 12:09:03.557604 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:03.557563    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s" event={"ID":"1e181569-4b01-4afe-9d02-278ff66af4ac","Type":"ContainerDied","Data":"a25837e3d41728c482673b3118b65c00e2491104ce4b2b7d12e279887b5c47a4"}
Nov 02 12:09:03.558033 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:03.558010    4942 scope.go:122] "RemoveContainer" containerID="a25837e3d41728c482673b3118b65c00e2491104ce4b2b7d12e279887b5c47a4"
Nov 02 12:09:03.559593 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:03.559559    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:09:03.559699 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:03.559633    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:09:03.559699 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:03.559668    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:09:04.638325 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:04.637907    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s" event={"ID":"1e181569-4b01-4afe-9d02-278ff66af4ac","Type":"ContainerStarted","Data":"924d5c2ed796af17cdea7157f2633605f38b383b48650d8a06c023903a0776cc"}
Nov 02 12:09:04.814322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:04.814144    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:09:04.834325 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:04.814796    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 02 12:09:04.834325 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:04.815233    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s" podUID="1e181569-4b01-4afe-9d02-278ff66af4ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 02 12:09:05.790741 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:05.790644    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 02 12:09:05.791317 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:05.791262    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s" podUID="1e181569-4b01-4afe-9d02-278ff66af4ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 02 12:09:06.789683 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:06.789645    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 02 12:09:06.789984 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:06.789953    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s" podUID="1e181569-4b01-4afe-9d02-278ff66af4ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 02 12:09:07.790871 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:07.790196    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 02 12:09:07.790871 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:07.790259    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s" podUID="1e181569-4b01-4afe-9d02-278ff66af4ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 02 12:09:08.788829 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:08.788789    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 02 12:09:08.789060 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:08.788845    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s" podUID="1e181569-4b01-4afe-9d02-278ff66af4ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 02 12:09:09.140796 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.140762    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"]
Nov 02 12:09:09.190700 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.190654    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.211566 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.211532    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"]
Nov 02 12:09:09.251400 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.251053    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-4b9210\"/\"simulator-w3-4b9210-kserve-self-signed-certs\""
Nov 02 12:09:09.293261 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.293139    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-dshm\") pod \"simulator-w3-4b9210-kserve-5548b779d-wqg5f\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") " pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.293261 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.293221    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-home\") pod \"simulator-w3-4b9210-kserve-5548b779d-wqg5f\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") " pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.293603 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.293272    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-model-cache\") pod \"simulator-w3-4b9210-kserve-5548b779d-wqg5f\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") " pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.293603 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.293367    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-tmp-dir\") pod \"simulator-w3-4b9210-kserve-5548b779d-wqg5f\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") " pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.293603 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.293397    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-tls-certs\") pod \"simulator-w3-4b9210-kserve-5548b779d-wqg5f\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") " pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.293603 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.293450    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwk29\" (UniqueName: \"kubernetes.io/projected/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-kube-api-access-nwk29\") pod \"simulator-w3-4b9210-kserve-5548b779d-wqg5f\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") " pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.293880 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.293855    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-4b9210\"/\"default-dockercfg-6254k\""
Nov 02 12:09:09.294606 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.294584    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-4b9210\"/\"openshift-service-ca.crt\""
Nov 02 12:09:09.294714 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.294689    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-4b9210\"/\"kube-root-ca.crt\""
Nov 02 12:09:09.395001 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.394903    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-model-cache\") pod \"simulator-w3-4b9210-kserve-5548b779d-wqg5f\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") " pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.395273 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.395250    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-tmp-dir\") pod \"simulator-w3-4b9210-kserve-5548b779d-wqg5f\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") " pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.395475 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.395456    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-tls-certs\") pod \"simulator-w3-4b9210-kserve-5548b779d-wqg5f\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") " pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.395796 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.395705    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwk29\" (UniqueName: \"kubernetes.io/projected/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-kube-api-access-nwk29\") pod \"simulator-w3-4b9210-kserve-5548b779d-wqg5f\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") " pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.396187 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.395975    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-model-cache\") pod \"simulator-w3-4b9210-kserve-5548b779d-wqg5f\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") " pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.396277 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.396185    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-tmp-dir\") pod \"simulator-w3-4b9210-kserve-5548b779d-wqg5f\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") " pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.396467 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.396377    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-dshm\") pod \"simulator-w3-4b9210-kserve-5548b779d-wqg5f\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") " pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.396592 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.396453    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-home\") pod \"simulator-w3-4b9210-kserve-5548b779d-wqg5f\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") " pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.396942 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.396921    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-home\") pod \"simulator-w3-4b9210-kserve-5548b779d-wqg5f\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") " pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.398639 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.398559    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-dshm\") pod \"simulator-w3-4b9210-kserve-5548b779d-wqg5f\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") " pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.399116 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.399092    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-tls-certs\") pod \"simulator-w3-4b9210-kserve-5548b779d-wqg5f\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") " pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.469102 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.469061    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwk29\" (UniqueName: \"kubernetes.io/projected/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-kube-api-access-nwk29\") pod \"simulator-w3-4b9210-kserve-5548b779d-wqg5f\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") " pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.528223 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.528179    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:09.790321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.789707    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Nov 02 12:09:09.790321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.789778    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s" podUID="1e181569-4b01-4afe-9d02-278ff66af4ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Nov 02 12:09:09.999870 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:09.999774    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"]
Nov 02 12:09:10.743505 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:10.743472    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f" event={"ID":"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212","Type":"ContainerStarted","Data":"d2cc63bd719bd45f72c078595901861432113fd272e2339f8112e595c4efc9bc"}
Nov 02 12:09:10.743505 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:10.743513    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f" event={"ID":"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212","Type":"ContainerStarted","Data":"59023a1b1fcc82fdd6d283e214f5b100a070f82e3abeb0267360d0bed8d88c59"}
Nov 02 12:09:10.811342 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:10.810497    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:09:10.815320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:10.811854    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:09:10.815320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:10.812670    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:09:10.863623 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:10.863561    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f" podStartSLOduration=1.863544506 podStartE2EDuration="1.863544506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:09:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:09:10.859393772 +0000 UTC m=+777.430743535" watchObservedRunningTime="2026-11-02 12:09:10.863544506 +0000 UTC m=+777.434894266"
Nov 02 12:09:11.528884 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:11.528842    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:14.431729 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.431696    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"]
Nov 02 12:09:14.435525 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.435504    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.454198 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.454166    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.454356 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.454211    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.454356 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.454294    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.454451 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.454403    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.454517 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.454499    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.454559 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.454526    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.454559 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.454546    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxj47\" (UniqueName: \"kubernetes.io/projected/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-kube-api-access-zxj47\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.454667 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.454652    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.454704 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.454683    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.541002 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.540963    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"]
Nov 02 12:09:14.555414 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.555381    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.555614 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.555442    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.555614 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.555472    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.555614 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.555502    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.555614 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.555570    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.555848 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.555637    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.555848 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.555653    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.555848 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.555680    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxj47\" (UniqueName: \"kubernetes.io/projected/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-kube-api-access-zxj47\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.555848 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.555718    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.556054 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.555981    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.556054 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.556001    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.556397 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.556372    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.556710 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.556686    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.557330 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.557289    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.558247 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.558225    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.558680 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.558660    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.616675 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.616638    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.616824 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.616642    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxj47\" (UniqueName: \"kubernetes.io/projected/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-kube-api-access-zxj47\") pod \"maas-default-gateway-openshift-default-6dcf757899-jhwsm\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:14.748782 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:14.748704    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:15.178070 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.178043    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"]
Nov 02 12:09:15.179735 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:09:15.179696    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod216e3c9b_1cf3_4b1e_9d49_a9ce3a3e5aa1.slice/crio-07b65172e1c2e70ba617b4fd5e8a9207ea58ea9a10a60bac62047c6cbc5ee9ca WatchSource:0}: Error finding container 07b65172e1c2e70ba617b4fd5e8a9207ea58ea9a10a60bac62047c6cbc5ee9ca: Status 404 returned error can't find the container with id 07b65172e1c2e70ba617b4fd5e8a9207ea58ea9a10a60bac62047c6cbc5ee9ca
Nov 02 12:09:15.182156 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.182120    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:09:15.182257 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.182182    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:09:15.182257 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.182215    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:09:15.567636 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.567566    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"]
Nov 02 12:09:15.571137 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.571120    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.625038 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.623886    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w4-dd087a-openshift-default-dockercfg-7896p\""
Nov 02 12:09:15.655129 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.655096    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"]
Nov 02 12:09:15.667079 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.667041    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-656ms\" (UniqueName: \"kubernetes.io/projected/c221cddd-0f93-4f81-9b40-1d017ad063fe-kube-api-access-656ms\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.667269 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.667107    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-workload-certs\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.667269 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.667212    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-workload-socket\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.667440 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.667289    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-data\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.667440 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.667387    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-credential-socket\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.667554 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.667444    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-token\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.667554 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.667490    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-envoy\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.667676 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.667540    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c221cddd-0f93-4f81-9b40-1d017ad063fe-istiod-ca-cert\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.667676 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.667633    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-podinfo\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.773320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.771549    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-workload-certs\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.773320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.772119    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-workload-socket\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.773320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.772485    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-workload-socket\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.773320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.772973    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-data\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.773320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.773289    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-data\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.773777 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.773559    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-credential-socket\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.795350 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.773880    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-credential-socket\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.795350 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.777644    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-token\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.795350 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.777700    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-envoy\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.795350 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.777753    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c221cddd-0f93-4f81-9b40-1d017ad063fe-istiod-ca-cert\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.795350 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.778255    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-podinfo\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.795350 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.778344    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-656ms\" (UniqueName: \"kubernetes.io/projected/c221cddd-0f93-4f81-9b40-1d017ad063fe-kube-api-access-656ms\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.795350 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.781124    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-envoy\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.795350 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.793969    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c221cddd-0f93-4f81-9b40-1d017ad063fe-istiod-ca-cert\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.795350 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.794089    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-workload-certs\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.795350 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.794497    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-podinfo\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.817008 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.816971    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm" event={"ID":"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1","Type":"ContainerStarted","Data":"28971369390a1457b619f972cbc11d58eaf2c0e10b864f0591a9fa08d0880822"}
Nov 02 12:09:15.817182 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.817063    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm" event={"ID":"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1","Type":"ContainerStarted","Data":"07b65172e1c2e70ba617b4fd5e8a9207ea58ea9a10a60bac62047c6cbc5ee9ca"}
Nov 02 12:09:15.839702 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.839624    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-token\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.841596 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.841569    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-656ms\" (UniqueName: \"kubernetes.io/projected/c221cddd-0f93-4f81-9b40-1d017ad063fe-kube-api-access-656ms\") pod \"e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") " pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.888706 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.888650    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:15.945860 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:15.945810    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm" podStartSLOduration=1.945792835 podStartE2EDuration="1.945792835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:09:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:09:15.94371272 +0000 UTC m=+782.515062474" watchObservedRunningTime="2026-11-02 12:09:15.945792835 +0000 UTC m=+782.517142592"
Nov 02 12:09:16.321109 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:16.321077    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"]
Nov 02 12:09:16.322077 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:09:16.322043    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc221cddd_0f93_4f81_9b40_1d017ad063fe.slice/crio-c69df4c20b6d2ad8afe5bf2ac1dad93d1a7d7966aa11b0a2bd4d2e00592adb02 WatchSource:0}: Error finding container c69df4c20b6d2ad8afe5bf2ac1dad93d1a7d7966aa11b0a2bd4d2e00592adb02: Status 404 returned error can't find the container with id c69df4c20b6d2ad8afe5bf2ac1dad93d1a7d7966aa11b0a2bd4d2e00592adb02
Nov 02 12:09:16.324441 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:16.324405    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:09:16.324547 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:16.324493    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:09:16.324610 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:16.324547    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:09:16.749836 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:16.749796    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:16.751186 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:16.751148    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-jhwsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 02 12:09:16.751354 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:16.751213    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm" podUID="216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 02 12:09:16.827761 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:16.827720    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz" event={"ID":"c221cddd-0f93-4f81-9b40-1d017ad063fe","Type":"ContainerStarted","Data":"6cfae5db3ae651812b857994c0b949e0f2232b1597102c5cc7c9af5a79fdcf5d"}
Nov 02 12:09:16.827961 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:16.827788    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz" event={"ID":"c221cddd-0f93-4f81-9b40-1d017ad063fe","Type":"ContainerStarted","Data":"c69df4c20b6d2ad8afe5bf2ac1dad93d1a7d7966aa11b0a2bd4d2e00592adb02"}
Nov 02 12:09:16.890322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:16.889482    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:16.960320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:16.952612    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz" podStartSLOduration=1.952591435 podStartE2EDuration="1.952591435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:09:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:09:16.951916377 +0000 UTC m=+783.523266134" watchObservedRunningTime="2026-11-02 12:09:16.952591435 +0000 UTC m=+783.523941195"
Nov 02 12:09:17.749471 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:17.749418    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-jhwsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 02 12:09:17.749471 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:17.749484    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm" podUID="216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 02 12:09:17.900919 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:17.900881    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:18.749486 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:18.749446    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-jhwsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 02 12:09:18.749689 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:18.749523    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm" podUID="216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 02 12:09:18.838584 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:18.838547    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:18.839429 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:18.839406    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:09:19.529139 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:19.529107    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:19.542796 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:19.542773    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:19.749203 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:19.749164    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-jhwsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 02 12:09:19.749387 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:19.749218    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm" podUID="216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 02 12:09:19.885894 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:19.885852    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:09:20.749401 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:20.749359    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-jhwsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 02 12:09:20.749772 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:20.749423    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm" podUID="216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 02 12:09:21.749603 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:21.749566    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-jhwsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 02 12:09:21.750015 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:21.749629    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm" podUID="216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 02 12:09:22.750083 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:22.750046    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-jhwsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Nov 02 12:09:22.750496 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:22.750113    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm" podUID="216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Nov 02 12:09:24.594890 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:24.594860    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:24.595413 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:24.595389    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:24.596240 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:24.596223    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:09:26.918786 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:26.918755    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h"]
Nov 02 12:09:26.923561 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:26.923541    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h"
Nov 02 12:09:26.981478 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:26.981451    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h"]
Nov 02 12:09:27.044140 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:27.042481    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc6f2\" (UniqueName: \"kubernetes.io/projected/48eaad6f-ce18-4218-a7c6-878ae119505a-kube-api-access-sc6f2\") pod \"payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h\" (UID: \"48eaad6f-ce18-4218-a7c6-878ae119505a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h"
Nov 02 12:09:27.044140 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:27.042577    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48eaad6f-ce18-4218-a7c6-878ae119505a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h\" (UID: \"48eaad6f-ce18-4218-a7c6-878ae119505a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h"
Nov 02 12:09:27.047509 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:27.047475    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w4-dd087a-dockercfg-nmd9w\""
Nov 02 12:09:27.047661 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:27.047572    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w4-dd087a\""
Nov 02 12:09:27.143688 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:27.143652    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc6f2\" (UniqueName: \"kubernetes.io/projected/48eaad6f-ce18-4218-a7c6-878ae119505a-kube-api-access-sc6f2\") pod \"payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h\" (UID: \"48eaad6f-ce18-4218-a7c6-878ae119505a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h"
Nov 02 12:09:27.143864 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:27.143752    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48eaad6f-ce18-4218-a7c6-878ae119505a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h\" (UID: \"48eaad6f-ce18-4218-a7c6-878ae119505a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h"
Nov 02 12:09:27.144480 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:27.144460    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48eaad6f-ce18-4218-a7c6-878ae119505a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h\" (UID: \"48eaad6f-ce18-4218-a7c6-878ae119505a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h"
Nov 02 12:09:27.231702 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:27.231631    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc6f2\" (UniqueName: \"kubernetes.io/projected/48eaad6f-ce18-4218-a7c6-878ae119505a-kube-api-access-sc6f2\") pod \"payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h\" (UID: \"48eaad6f-ce18-4218-a7c6-878ae119505a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h"
Nov 02 12:09:27.241026 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:27.240995    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h"
Nov 02 12:09:27.622493 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:27.622470    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h"]
Nov 02 12:09:27.624062 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:09:27.624034    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48eaad6f_ce18_4218_a7c6_878ae119505a.slice/crio-d31816409b33445ea1f4f17603d314cc7f59687224e80e81d1739fc9d732a864 WatchSource:0}: Error finding container d31816409b33445ea1f4f17603d314cc7f59687224e80e81d1739fc9d732a864: Status 404 returned error can't find the container with id d31816409b33445ea1f4f17603d314cc7f59687224e80e81d1739fc9d732a864
Nov 02 12:09:27.893930 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:27.893841    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h" event={"ID":"48eaad6f-ce18-4218-a7c6-878ae119505a","Type":"ContainerStarted","Data":"b1998864e4aae4b0d879b2abd0409597897350b40f62ca902a986a9096fdfb5c"}
Nov 02 12:09:27.893930 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:27.893892    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h" event={"ID":"48eaad6f-ce18-4218-a7c6-878ae119505a","Type":"ContainerStarted","Data":"d31816409b33445ea1f4f17603d314cc7f59687224e80e81d1739fc9d732a864"}
Nov 02 12:09:27.894117 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:27.893963    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h"
Nov 02 12:09:28.010454 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:28.010399    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h" podStartSLOduration=2.01038464 podStartE2EDuration="2.01038464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:09:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:09:28.008663766 +0000 UTC m=+794.580013527" watchObservedRunningTime="2026-11-02 12:09:28.01038464 +0000 UTC m=+794.581734422"
Nov 02 12:09:30.981786 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:09:30.981751    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:09:38.903565 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:38.903533    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h"
Nov 02 12:09:39.071634 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:39.071600    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"]
Nov 02 12:09:39.076278 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:39.076251    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:09:39.133234 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:39.133201    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-966817-serving-cert\""
Nov 02 12:09:39.136771 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:39.136743    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"]
Nov 02 12:09:39.176429 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:39.176338    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6712798b-0759-43a3-a5c6-a6c475ebc228-maas-api-tls\") pod \"maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw\" (UID: \"6712798b-0759-43a3-a5c6-a6c475ebc228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:09:39.176591 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:39.176498    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6712798b-0759-43a3-a5c6-a6c475ebc228-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw\" (UID: \"6712798b-0759-43a3-a5c6-a6c475ebc228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:09:39.176591 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:39.176554    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6712798b-0759-43a3-a5c6-a6c475ebc228-metrics-certs\") pod \"maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw\" (UID: \"6712798b-0759-43a3-a5c6-a6c475ebc228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:09:39.176703 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:39.176594    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h72x\" (UniqueName: \"kubernetes.io/projected/6712798b-0759-43a3-a5c6-a6c475ebc228-kube-api-access-7h72x\") pod \"maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw\" (UID: \"6712798b-0759-43a3-a5c6-a6c475ebc228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:09:39.282500 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:39.282457    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6712798b-0759-43a3-a5c6-a6c475ebc228-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw\" (UID: \"6712798b-0759-43a3-a5c6-a6c475ebc228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:09:39.282685 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:39.282532    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6712798b-0759-43a3-a5c6-a6c475ebc228-metrics-certs\") pod \"maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw\" (UID: \"6712798b-0759-43a3-a5c6-a6c475ebc228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:09:39.282685 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:39.282591    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h72x\" (UniqueName: \"kubernetes.io/projected/6712798b-0759-43a3-a5c6-a6c475ebc228-kube-api-access-7h72x\") pod \"maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw\" (UID: \"6712798b-0759-43a3-a5c6-a6c475ebc228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:09:39.282685 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:39.282669    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6712798b-0759-43a3-a5c6-a6c475ebc228-maas-api-tls\") pod \"maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw\" (UID: \"6712798b-0759-43a3-a5c6-a6c475ebc228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:09:39.282920 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:09:39.282900    4942 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-serving-cert: secret "maas-api-e2e-praxis-b-966817-serving-cert" not found
Nov 02 12:09:39.283075 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:09:39.283058    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6712798b-0759-43a3-a5c6-a6c475ebc228-maas-api-tls podName:6712798b-0759-43a3-a5c6-a6c475ebc228 nodeName:}" failed. No retries permitted until 2026-11-02 12:09:39.783034252 +0000 UTC m=+806.354384003 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6712798b-0759-43a3-a5c6-a6c475ebc228-maas-api-tls") pod "maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw" (UID: "6712798b-0759-43a3-a5c6-a6c475ebc228") : secret "maas-api-e2e-praxis-b-966817-serving-cert" not found
Nov 02 12:09:39.288503 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:39.287573    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6712798b-0759-43a3-a5c6-a6c475ebc228-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw\" (UID: \"6712798b-0759-43a3-a5c6-a6c475ebc228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:09:39.288503 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:39.288472    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6712798b-0759-43a3-a5c6-a6c475ebc228-metrics-certs\") pod \"maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw\" (UID: \"6712798b-0759-43a3-a5c6-a6c475ebc228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:09:39.346856 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:39.346823    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h72x\" (UniqueName: \"kubernetes.io/projected/6712798b-0759-43a3-a5c6-a6c475ebc228-kube-api-access-7h72x\") pod \"maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw\" (UID: \"6712798b-0759-43a3-a5c6-a6c475ebc228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:09:39.789881 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:39.789231    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6712798b-0759-43a3-a5c6-a6c475ebc228-maas-api-tls\") pod \"maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw\" (UID: \"6712798b-0759-43a3-a5c6-a6c475ebc228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:09:39.798742 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:39.798712    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6712798b-0759-43a3-a5c6-a6c475ebc228-maas-api-tls\") pod \"maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw\" (UID: \"6712798b-0759-43a3-a5c6-a6c475ebc228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:09:40.001622 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:40.001587    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:09:40.418768 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:40.418737    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"]
Nov 02 12:09:40.421249 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:09:40.421223    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6712798b_0759_43a3_a5c6_a6c475ebc228.slice/crio-3c6c499dc009fa6a92d3dab486a9fd2e2f05df4cc67f067bfd3535b50ab023ad WatchSource:0}: Error finding container 3c6c499dc009fa6a92d3dab486a9fd2e2f05df4cc67f067bfd3535b50ab023ad: Status 404 returned error can't find the container with id 3c6c499dc009fa6a92d3dab486a9fd2e2f05df4cc67f067bfd3535b50ab023ad
Nov 02 12:09:40.972866 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:40.972832    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw" event={"ID":"6712798b-0759-43a3-a5c6-a6c475ebc228","Type":"ContainerStarted","Data":"3c6c499dc009fa6a92d3dab486a9fd2e2f05df4cc67f067bfd3535b50ab023ad"}
Nov 02 12:09:41.979970 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:41.979929    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw" event={"ID":"6712798b-0759-43a3-a5c6-a6c475ebc228","Type":"ContainerStarted","Data":"b12ceae0484ca2294cf2e91ef6f9da27cabde6889f78a2f317da79b385bbe263"}
Nov 02 12:09:41.980457 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:41.980131    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:09:42.098764 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:42.098713    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw" podStartSLOduration=3.163050644 podStartE2EDuration="4.0986988s" totalImagesPullingTime="935.648156ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:09:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:09:42.096418738 +0000 UTC m=+808.667768508" watchObservedRunningTime="2026-11-02 12:09:42.0986988 +0000 UTC m=+808.670048557"
Nov 02 12:09:47.992568 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:47.992535    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:09:49.036334 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:49.036286    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"]
Nov 02 12:09:49.040722 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:49.040705    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:09:49.091187 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:49.091154    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97173ac8-689b-41ea-9e10-f99054fae648-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc\" (UID: \"97173ac8-689b-41ea-9e10-f99054fae648\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:09:49.091360 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:49.091197    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97173ac8-689b-41ea-9e10-f99054fae648-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc\" (UID: \"97173ac8-689b-41ea-9e10-f99054fae648\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:09:49.091360 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:49.091306    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97173ac8-689b-41ea-9e10-f99054fae648-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc\" (UID: \"97173ac8-689b-41ea-9e10-f99054fae648\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:09:49.091360 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:49.091350    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc6q4\" (UniqueName: \"kubernetes.io/projected/97173ac8-689b-41ea-9e10-f99054fae648-kube-api-access-hc6q4\") pod \"maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc\" (UID: \"97173ac8-689b-41ea-9e10-f99054fae648\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:09:49.092923 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:49.092902    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w4-dd087a-serving-cert\""
Nov 02 12:09:49.145069 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:49.145030    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"]
Nov 02 12:09:49.192053 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:49.192020    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97173ac8-689b-41ea-9e10-f99054fae648-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc\" (UID: \"97173ac8-689b-41ea-9e10-f99054fae648\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:09:49.192240 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:49.192062    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97173ac8-689b-41ea-9e10-f99054fae648-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc\" (UID: \"97173ac8-689b-41ea-9e10-f99054fae648\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:09:49.192240 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:49.192104    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97173ac8-689b-41ea-9e10-f99054fae648-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc\" (UID: \"97173ac8-689b-41ea-9e10-f99054fae648\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:09:49.192240 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:49.192130    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc6q4\" (UniqueName: \"kubernetes.io/projected/97173ac8-689b-41ea-9e10-f99054fae648-kube-api-access-hc6q4\") pod \"maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc\" (UID: \"97173ac8-689b-41ea-9e10-f99054fae648\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:09:49.192240 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:09:49.192182    4942 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-serving-cert: secret "maas-api-e2e-shared-a-w4-dd087a-serving-cert" not found
Nov 02 12:09:49.192487 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:09:49.192263    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97173ac8-689b-41ea-9e10-f99054fae648-maas-api-tls podName:97173ac8-689b-41ea-9e10-f99054fae648 nodeName:}" failed. No retries permitted until 2026-11-02 12:09:49.692242566 +0000 UTC m=+816.263592315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/97173ac8-689b-41ea-9e10-f99054fae648-maas-api-tls") pod "maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc" (UID: "97173ac8-689b-41ea-9e10-f99054fae648") : secret "maas-api-e2e-shared-a-w4-dd087a-serving-cert" not found
Nov 02 12:09:49.192920 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:49.192887    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97173ac8-689b-41ea-9e10-f99054fae648-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc\" (UID: \"97173ac8-689b-41ea-9e10-f99054fae648\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:09:49.194693 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:49.194668    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97173ac8-689b-41ea-9e10-f99054fae648-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc\" (UID: \"97173ac8-689b-41ea-9e10-f99054fae648\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:09:49.253223 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:49.253192    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc6q4\" (UniqueName: \"kubernetes.io/projected/97173ac8-689b-41ea-9e10-f99054fae648-kube-api-access-hc6q4\") pod \"maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc\" (UID: \"97173ac8-689b-41ea-9e10-f99054fae648\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:09:49.707211 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:49.707175    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97173ac8-689b-41ea-9e10-f99054fae648-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc\" (UID: \"97173ac8-689b-41ea-9e10-f99054fae648\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:09:49.709982 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:49.709956    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97173ac8-689b-41ea-9e10-f99054fae648-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc\" (UID: \"97173ac8-689b-41ea-9e10-f99054fae648\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:09:49.955033 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:49.954990    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:09:50.341130 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:50.341037    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"]
Nov 02 12:09:50.348167 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:09:50.348134    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97173ac8_689b_41ea_9e10_f99054fae648.slice/crio-6f42b58436c003d3596783e6db32d497f0745a41681de6ac27705f2473abe354 WatchSource:0}: Error finding container 6f42b58436c003d3596783e6db32d497f0745a41681de6ac27705f2473abe354: Status 404 returned error can't find the container with id 6f42b58436c003d3596783e6db32d497f0745a41681de6ac27705f2473abe354
Nov 02 12:09:51.043970 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:51.043933    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc" event={"ID":"97173ac8-689b-41ea-9e10-f99054fae648","Type":"ContainerStarted","Data":"6f42b58436c003d3596783e6db32d497f0745a41681de6ac27705f2473abe354"}
Nov 02 12:09:52.051931 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:52.051898    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc" event={"ID":"97173ac8-689b-41ea-9e10-f99054fae648","Type":"ContainerStarted","Data":"6a78bb58c1a760e93ccc67cabaa1281671c2fd0f0d8c559d5a8781b2bb37603c"}
Nov 02 12:09:52.052434 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:52.051993    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:09:52.173688 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:52.173634    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc" podStartSLOduration=3.348089915 podStartE2EDuration="4.173619577s" totalImagesPullingTime="825.529662ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:09:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:09:52.171118944 +0000 UTC m=+818.742468702" watchObservedRunningTime="2026-11-02 12:09:52.173619577 +0000 UTC m=+818.744969333"
Nov 02 12:09:58.063572 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:09:58.063542    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:10:01.001618 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:10:01.001575    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:10:10.165787 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:10.165754    4942 generic.go:356] "Generic (PLEG): container finished" podID="c221cddd-0f93-4f81-9b40-1d017ad063fe" containerID="6cfae5db3ae651812b857994c0b949e0f2232b1597102c5cc7c9af5a79fdcf5d" exitCode=0
Nov 02 12:10:10.166211 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:10.165835    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz" event={"ID":"c221cddd-0f93-4f81-9b40-1d017ad063fe","Type":"ContainerDied","Data":"6cfae5db3ae651812b857994c0b949e0f2232b1597102c5cc7c9af5a79fdcf5d"}
Nov 02 12:10:10.166286 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:10.166271    4942 scope.go:122] "RemoveContainer" containerID="6cfae5db3ae651812b857994c0b949e0f2232b1597102c5cc7c9af5a79fdcf5d"
Nov 02 12:10:10.167722 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:10.167691    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:10:10.167796 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:10.167773    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:10:10.167839 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:10.167820    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:10:11.184605 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:11.184566    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz" event={"ID":"c221cddd-0f93-4f81-9b40-1d017ad063fe","Type":"ContainerStarted","Data":"8e88d51026c9072b07b1018d80ea26b7b3493617ea8abbacf19d133498a8b794"}
Nov 02 12:10:11.889904 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:11.889811    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:10:11.890107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:11.890035    4942 patch_prober.go:29] interesting pod/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 02 12:10:11.890107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:11.890088    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz" podUID="c221cddd-0f93-4f81-9b40-1d017ad063fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 02 12:10:12.890610 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:12.890574    4942 patch_prober.go:29] interesting pod/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 02 12:10:12.891118 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:12.890685    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz" podUID="c221cddd-0f93-4f81-9b40-1d017ad063fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 02 12:10:13.889409 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:13.889379    4942 patch_prober.go:29] interesting pod/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 02 12:10:13.889683 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:13.889658    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz" podUID="c221cddd-0f93-4f81-9b40-1d017ad063fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 02 12:10:14.890353 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:14.890316    4942 patch_prober.go:29] interesting pod/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 02 12:10:14.890738 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:14.890390    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz" podUID="c221cddd-0f93-4f81-9b40-1d017ad063fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 02 12:10:15.889741 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:15.889693    4942 patch_prober.go:29] interesting pod/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Nov 02 12:10:15.889948 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:15.889769    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz" podUID="c221cddd-0f93-4f81-9b40-1d017ad063fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Nov 02 12:10:16.893481 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:16.893448    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:10:16.893933 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:16.893750    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:10:16.894365 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:16.894350    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:10:30.981970 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:10:30.981893    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:10:35.279367 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:35.279330    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png"]
Nov 02 12:10:35.283093 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:35.283076    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png"
Nov 02 12:10:35.342857 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:35.342756    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png"]
Nov 02 12:10:35.343587 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:35.343553    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2z4c\" (UniqueName: \"kubernetes.io/projected/32bcdc82-f897-4744-a140-f96f352a3422-kube-api-access-r2z4c\") pod \"payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png\" (UID: \"32bcdc82-f897-4744-a140-f96f352a3422\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png"
Nov 02 12:10:35.343712 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:35.343632    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32bcdc82-f897-4744-a140-f96f352a3422-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png\" (UID: \"32bcdc82-f897-4744-a140-f96f352a3422\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png"
Nov 02 12:10:35.386110 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:35.386076    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w4-03c4e8\""
Nov 02 12:10:35.386365 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:35.386190    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w4-03c4e8-dockercfg-nd29c\""
Nov 02 12:10:35.445374 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:35.445340    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2z4c\" (UniqueName: \"kubernetes.io/projected/32bcdc82-f897-4744-a140-f96f352a3422-kube-api-access-r2z4c\") pod \"payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png\" (UID: \"32bcdc82-f897-4744-a140-f96f352a3422\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png"
Nov 02 12:10:35.445547 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:35.445394    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32bcdc82-f897-4744-a140-f96f352a3422-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png\" (UID: \"32bcdc82-f897-4744-a140-f96f352a3422\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png"
Nov 02 12:10:35.446076 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:35.446055    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32bcdc82-f897-4744-a140-f96f352a3422-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png\" (UID: \"32bcdc82-f897-4744-a140-f96f352a3422\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png"
Nov 02 12:10:35.506094 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:35.506064    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2z4c\" (UniqueName: \"kubernetes.io/projected/32bcdc82-f897-4744-a140-f96f352a3422-kube-api-access-r2z4c\") pod \"payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png\" (UID: \"32bcdc82-f897-4744-a140-f96f352a3422\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png"
Nov 02 12:10:35.596736 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:35.596658    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png"
Nov 02 12:10:35.971752 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:35.971725    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png"]
Nov 02 12:10:35.973383 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:10:35.973356    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32bcdc82_f897_4744_a140_f96f352a3422.slice/crio-b7880bcea0ad5f0a399ee672c137b267115e535de0c4f1ab3e436d246503a248 WatchSource:0}: Error finding container b7880bcea0ad5f0a399ee672c137b267115e535de0c4f1ab3e436d246503a248: Status 404 returned error can't find the container with id b7880bcea0ad5f0a399ee672c137b267115e535de0c4f1ab3e436d246503a248
Nov 02 12:10:36.365385 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:36.365342    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png" event={"ID":"32bcdc82-f897-4744-a140-f96f352a3422","Type":"ContainerStarted","Data":"48144bce98a2eb6f3918ade653c0a794ceda0a52b27eac34dc359ad18fecc385"}
Nov 02 12:10:36.365792 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:36.365411    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png" event={"ID":"32bcdc82-f897-4744-a140-f96f352a3422","Type":"ContainerStarted","Data":"b7880bcea0ad5f0a399ee672c137b267115e535de0c4f1ab3e436d246503a248"}
Nov 02 12:10:36.365792 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:36.365444    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png"
Nov 02 12:10:36.482458 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:36.482409    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png" podStartSLOduration=1.482395704 podStartE2EDuration="1.482395704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:10:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:10:36.480763027 +0000 UTC m=+863.052112786" watchObservedRunningTime="2026-11-02 12:10:36.482395704 +0000 UTC m=+863.053745460"
Nov 02 12:10:43.901119 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:43.901087    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7221-dcee02"]
Nov 02 12:10:43.905672 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:43.905640    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7221-dcee02"
Nov 02 12:10:43.961435 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:43.961406    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7221-dcee02"]
Nov 02 12:10:44.007658 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:44.007628    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5lqpk\""
Nov 02 12:10:44.032614 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:44.032573    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjhtr\" (UniqueName: \"kubernetes.io/projected/2ee219ad-334d-4366-8142-55b42198c763-kube-api-access-tjhtr\") pod \"test-curl-7221-dcee02\" (UID: \"2ee219ad-334d-4366-8142-55b42198c763\") " pod="opendatahub/test-curl-7221-dcee02"
Nov 02 12:10:44.134105 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:44.134070    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjhtr\" (UniqueName: \"kubernetes.io/projected/2ee219ad-334d-4366-8142-55b42198c763-kube-api-access-tjhtr\") pod \"test-curl-7221-dcee02\" (UID: \"2ee219ad-334d-4366-8142-55b42198c763\") " pod="opendatahub/test-curl-7221-dcee02"
Nov 02 12:10:44.195405 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:44.195329    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjhtr\" (UniqueName: \"kubernetes.io/projected/2ee219ad-334d-4366-8142-55b42198c763-kube-api-access-tjhtr\") pod \"test-curl-7221-dcee02\" (UID: \"2ee219ad-334d-4366-8142-55b42198c763\") " pod="opendatahub/test-curl-7221-dcee02"
Nov 02 12:10:44.219054 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:44.219026    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7221-dcee02"
Nov 02 12:10:44.607494 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:44.607461    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7221-dcee02"]
Nov 02 12:10:44.608919 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:10:44.608880    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ee219ad_334d_4366_8142_55b42198c763.slice/crio-27ff88550d541948c1c7dc4d74b82118a26f73abf5f5cb88fa27b8343c7f66f0 WatchSource:0}: Error finding container 27ff88550d541948c1c7dc4d74b82118a26f73abf5f5cb88fa27b8343c7f66f0: Status 404 returned error can't find the container with id 27ff88550d541948c1c7dc4d74b82118a26f73abf5f5cb88fa27b8343c7f66f0
Nov 02 12:10:45.419483 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:45.419442    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7221-dcee02" event={"ID":"2ee219ad-334d-4366-8142-55b42198c763","Type":"ContainerStarted","Data":"27ff88550d541948c1c7dc4d74b82118a26f73abf5f5cb88fa27b8343c7f66f0"}
Nov 02 12:10:47.374938 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:47.374906    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png"
Nov 02 12:10:47.434911 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:47.434874    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7221-dcee02" event={"ID":"2ee219ad-334d-4366-8142-55b42198c763","Type":"ContainerStarted","Data":"99018fb96ac51de4a3be54432ea4b22b6549ea3f90c20c625a0ab070e7bd2772"}
Nov 02 12:10:47.614071 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:47.614013    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7221-dcee02" podStartSLOduration=2.305110251 podStartE2EDuration="4.613995836s" totalImagesPullingTime="2.308885585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:10:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:10:47.61283696 +0000 UTC m=+874.184186734" watchObservedRunningTime="2026-11-02 12:10:47.613995836 +0000 UTC m=+874.185345634"
Nov 02 12:10:49.021315 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:49.021268    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7221-dcee02"]
Nov 02 12:10:49.024526 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:49.024498    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7221-dcee02"]
Nov 02 12:10:49.178044 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:49.178009    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7221-219746"]
Nov 02 12:10:49.178533 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:49.178517    4942 cpu_manager.go:414] "containerMap: removing container" podUID="2ee219ad-334d-4366-8142-55b42198c763" containerName="test-curl-7221-dcee02"
Nov 02 12:10:49.178584 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:49.178535    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ee219ad-334d-4366-8142-55b42198c763" containerName="test-curl-7221-dcee02"
Nov 02 12:10:49.178651 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:49.178642    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ee219ad-334d-4366-8142-55b42198c763" containerName="test-curl-7221-dcee02"
Nov 02 12:10:49.182662 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:49.182644    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7221-219746"
Nov 02 12:10:49.237924 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:49.237896    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7221-219746"]
Nov 02 12:10:49.292000 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:49.291925    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xhnz\" (UniqueName: \"kubernetes.io/projected/cb55b82c-d412-4191-9d66-94fe35bea569-kube-api-access-8xhnz\") pod \"test-curl-7221-219746\" (UID: \"cb55b82c-d412-4191-9d66-94fe35bea569\") " pod="opendatahub/test-curl-7221-219746"
Nov 02 12:10:49.392968 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:49.392930    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xhnz\" (UniqueName: \"kubernetes.io/projected/cb55b82c-d412-4191-9d66-94fe35bea569-kube-api-access-8xhnz\") pod \"test-curl-7221-219746\" (UID: \"cb55b82c-d412-4191-9d66-94fe35bea569\") " pod="opendatahub/test-curl-7221-219746"
Nov 02 12:10:49.453566 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:49.453526    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7221-dcee02" podUID="2ee219ad-334d-4366-8142-55b42198c763" containerName="test-curl-7221-dcee02" containerID="cri-o://99018fb96ac51de4a3be54432ea4b22b6549ea3f90c20c625a0ab070e7bd2772" gracePeriod=2
Nov 02 12:10:49.456154 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:49.456123    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xhnz\" (UniqueName: \"kubernetes.io/projected/cb55b82c-d412-4191-9d66-94fe35bea569-kube-api-access-8xhnz\") pod \"test-curl-7221-219746\" (UID: \"cb55b82c-d412-4191-9d66-94fe35bea569\") " pod="opendatahub/test-curl-7221-219746"
Nov 02 12:10:49.495052 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:49.495022    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7221-219746"
Nov 02 12:10:49.504404 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:10:49.504371    4942 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7221-dcee02\" is forbidden: User \"system:node:ip-10-0-135-24.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-24.us-west-1.compute.internal' and this object" podUID="2ee219ad-334d-4366-8142-55b42198c763" pod="opendatahub/test-curl-7221-dcee02"
Nov 02 12:10:49.874349 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:49.874099    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7221-219746"]
Nov 02 12:10:49.876736 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:10:49.876706    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb55b82c_d412_4191_9d66_94fe35bea569.slice/crio-146e5cf79cb1941003e75621ddf95b1718e86c48ba5ee09c5b1816f6cc11deb3 WatchSource:0}: Error finding container 146e5cf79cb1941003e75621ddf95b1718e86c48ba5ee09c5b1816f6cc11deb3: Status 404 returned error can't find the container with id 146e5cf79cb1941003e75621ddf95b1718e86c48ba5ee09c5b1816f6cc11deb3
Nov 02 12:10:50.467322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:50.467248    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7221-219746" event={"ID":"cb55b82c-d412-4191-9d66-94fe35bea569","Type":"ContainerStarted","Data":"7fdc2e622d5e63ae878012c9ba45c3f2216d440eb4d304082ed2bc21c3376103"}
Nov 02 12:10:50.467322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:50.467312    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7221-219746" event={"ID":"cb55b82c-d412-4191-9d66-94fe35bea569","Type":"ContainerStarted","Data":"146e5cf79cb1941003e75621ddf95b1718e86c48ba5ee09c5b1816f6cc11deb3"}
Nov 02 12:10:50.583147 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:50.583097    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7221-219746" podStartSLOduration=1.583083488 podStartE2EDuration="1.583083488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:10:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:10:50.582827816 +0000 UTC m=+877.154177573" watchObservedRunningTime="2026-11-02 12:10:50.583083488 +0000 UTC m=+877.154433264"
Nov 02 12:10:51.701803 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:51.701773    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7221-dcee02"
Nov 02 12:10:51.820199 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:51.820125    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ee219ad-334d-4366-8142-55b42198c763-kube-api-access-tjhtr\" (UniqueName: \"kubernetes.io/projected/2ee219ad-334d-4366-8142-55b42198c763-kube-api-access-tjhtr\") pod \"2ee219ad-334d-4366-8142-55b42198c763\" (UID: \"2ee219ad-334d-4366-8142-55b42198c763\") "
Nov 02 12:10:51.822159 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:51.822133    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ee219ad-334d-4366-8142-55b42198c763-kube-api-access-tjhtr" pod "2ee219ad-334d-4366-8142-55b42198c763" (UID: "2ee219ad-334d-4366-8142-55b42198c763"). InnerVolumeSpecName "kube-api-access-tjhtr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:10:51.921535 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:51.921505    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjhtr\" (UniqueName: \"kubernetes.io/projected/2ee219ad-334d-4366-8142-55b42198c763-kube-api-access-tjhtr\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:10:52.135768 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:52.135735    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7221-219746"]
Nov 02 12:10:52.141266 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:52.141235    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7221-219746"]
Nov 02 12:10:52.483466 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:52.483367    4942 generic.go:356] "Generic (PLEG): container finished" podID="2ee219ad-334d-4366-8142-55b42198c763" containerID="99018fb96ac51de4a3be54432ea4b22b6549ea3f90c20c625a0ab070e7bd2772" exitCode=137
Nov 02 12:10:52.483466 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:52.483424    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7221-dcee02"
Nov 02 12:10:52.483466 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:52.483454    4942 scope.go:122] "RemoveContainer" containerID="99018fb96ac51de4a3be54432ea4b22b6549ea3f90c20c625a0ab070e7bd2772"
Nov 02 12:10:52.483739 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:52.483690    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7221-219746" podUID="cb55b82c-d412-4191-9d66-94fe35bea569" containerName="test-curl-7221-219746" containerID="cri-o://7fdc2e622d5e63ae878012c9ba45c3f2216d440eb4d304082ed2bc21c3376103" gracePeriod=2
Nov 02 12:10:52.494561 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:52.494540    4942 scope.go:122] "RemoveContainer" containerID="99018fb96ac51de4a3be54432ea4b22b6549ea3f90c20c625a0ab070e7bd2772"
Nov 02 12:10:52.494839 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:10:52.494817    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99018fb96ac51de4a3be54432ea4b22b6549ea3f90c20c625a0ab070e7bd2772\": container with ID starting with 99018fb96ac51de4a3be54432ea4b22b6549ea3f90c20c625a0ab070e7bd2772 not found: ID does not exist" containerID="99018fb96ac51de4a3be54432ea4b22b6549ea3f90c20c625a0ab070e7bd2772"
Nov 02 12:10:52.494911 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:52.494851    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99018fb96ac51de4a3be54432ea4b22b6549ea3f90c20c625a0ab070e7bd2772"} err="failed to get container status \"99018fb96ac51de4a3be54432ea4b22b6549ea3f90c20c625a0ab070e7bd2772\": rpc error: code = NotFound desc = could not find container \"99018fb96ac51de4a3be54432ea4b22b6549ea3f90c20c625a0ab070e7bd2772\": container with ID starting with 99018fb96ac51de4a3be54432ea4b22b6549ea3f90c20c625a0ab070e7bd2772 not found: ID does not exist"
Nov 02 12:10:52.534535 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:10:52.534504    4942 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7221-219746\" is forbidden: User \"system:node:ip-10-0-135-24.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-24.us-west-1.compute.internal' and this object" podUID="cb55b82c-d412-4191-9d66-94fe35bea569" pod="opendatahub/test-curl-7221-219746"
Nov 02 12:10:52.585348 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:10:52.585279    4942 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7221-dcee02\" is forbidden: User \"system:node:ip-10-0-135-24.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-24.us-west-1.compute.internal' and this object" podUID="2ee219ad-334d-4366-8142-55b42198c763" pod="opendatahub/test-curl-7221-dcee02"
Nov 02 12:10:52.636123 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:10:52.636085    4942 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7221-219746\" is forbidden: User \"system:node:ip-10-0-135-24.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-24.us-west-1.compute.internal' and this object" podUID="cb55b82c-d412-4191-9d66-94fe35bea569" pod="opendatahub/test-curl-7221-219746"
Nov 02 12:10:53.589509 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:53.588942    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ee219ad-334d-4366-8142-55b42198c763" path="/var/lib/kubelet/pods/2ee219ad-334d-4366-8142-55b42198c763/volumes"
Nov 02 12:10:53.630871 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:10:53.630828    4942 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7221-219746\" is forbidden: User \"system:node:ip-10-0-135-24.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-24.us-west-1.compute.internal' and this object" podUID="cb55b82c-d412-4191-9d66-94fe35bea569" pod="opendatahub/test-curl-7221-219746"
Nov 02 12:10:53.683076 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:10:53.683031    4942 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7221-dcee02\" is forbidden: User \"system:node:ip-10-0-135-24.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-24.us-west-1.compute.internal' and this object" podUID="2ee219ad-334d-4366-8142-55b42198c763" pod="opendatahub/test-curl-7221-dcee02"
Nov 02 12:10:54.708664 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:54.708641    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7221-219746"
Nov 02 12:10:54.776835 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:54.776751    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb55b82c-d412-4191-9d66-94fe35bea569-kube-api-access-8xhnz\" (UniqueName: \"kubernetes.io/projected/cb55b82c-d412-4191-9d66-94fe35bea569-kube-api-access-8xhnz\") pod \"cb55b82c-d412-4191-9d66-94fe35bea569\" (UID: \"cb55b82c-d412-4191-9d66-94fe35bea569\") "
Nov 02 12:10:54.778857 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:54.778836    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb55b82c-d412-4191-9d66-94fe35bea569-kube-api-access-8xhnz" pod "cb55b82c-d412-4191-9d66-94fe35bea569" (UID: "cb55b82c-d412-4191-9d66-94fe35bea569"). InnerVolumeSpecName "kube-api-access-8xhnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:10:54.878268 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:54.878230    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xhnz\" (UniqueName: \"kubernetes.io/projected/cb55b82c-d412-4191-9d66-94fe35bea569-kube-api-access-8xhnz\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:10:55.504840 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:55.504804    4942 generic.go:356] "Generic (PLEG): container finished" podID="cb55b82c-d412-4191-9d66-94fe35bea569" containerID="7fdc2e622d5e63ae878012c9ba45c3f2216d440eb4d304082ed2bc21c3376103" exitCode=137
Nov 02 12:10:55.504998 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:55.504844    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7221-219746"
Nov 02 12:10:55.504998 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:55.504861    4942 scope.go:122] "RemoveContainer" containerID="7fdc2e622d5e63ae878012c9ba45c3f2216d440eb4d304082ed2bc21c3376103"
Nov 02 12:10:55.514067 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:55.514048    4942 scope.go:122] "RemoveContainer" containerID="7fdc2e622d5e63ae878012c9ba45c3f2216d440eb4d304082ed2bc21c3376103"
Nov 02 12:10:55.514342 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:10:55.514322    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fdc2e622d5e63ae878012c9ba45c3f2216d440eb4d304082ed2bc21c3376103\": container with ID starting with 7fdc2e622d5e63ae878012c9ba45c3f2216d440eb4d304082ed2bc21c3376103 not found: ID does not exist" containerID="7fdc2e622d5e63ae878012c9ba45c3f2216d440eb4d304082ed2bc21c3376103"
Nov 02 12:10:55.514385 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:55.514350    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fdc2e622d5e63ae878012c9ba45c3f2216d440eb4d304082ed2bc21c3376103"} err="failed to get container status \"7fdc2e622d5e63ae878012c9ba45c3f2216d440eb4d304082ed2bc21c3376103\": rpc error: code = NotFound desc = could not find container \"7fdc2e622d5e63ae878012c9ba45c3f2216d440eb4d304082ed2bc21c3376103\": container with ID starting with 7fdc2e622d5e63ae878012c9ba45c3f2216d440eb4d304082ed2bc21c3376103 not found: ID does not exist"
Nov 02 12:10:55.542093 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:10:55.542069    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cb55b82c-d412-4191-9d66-94fe35bea569" path="/var/lib/kubelet/pods/cb55b82c-d412-4191-9d66-94fe35bea569/volumes"
Nov 02 12:10:55.564162 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:10:55.564133    4942 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7221-219746\" is forbidden: User \"system:node:ip-10-0-135-24.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-24.us-west-1.compute.internal' and this object" podUID="cb55b82c-d412-4191-9d66-94fe35bea569" pod="opendatahub/test-curl-7221-219746"
Nov 02 12:11:00.982430 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:11:00.982390    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:11:01.616403 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.616367    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84"]
Nov 02 12:11:01.616940 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.616916    4942 cpu_manager.go:414] "containerMap: removing container" podUID="cb55b82c-d412-4191-9d66-94fe35bea569" containerName="test-curl-7221-219746"
Nov 02 12:11:01.616940 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.616941    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb55b82c-d412-4191-9d66-94fe35bea569" containerName="test-curl-7221-219746"
Nov 02 12:11:01.617099 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.617060    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb55b82c-d412-4191-9d66-94fe35bea569" containerName="test-curl-7221-219746"
Nov 02 12:11:01.620388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.620373    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84"
Nov 02 12:11:01.648820 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.648794    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04230821-2095-474d-aa72-8dc32b08ca0f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84\" (UID: \"04230821-2095-474d-aa72-8dc32b08ca0f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84"
Nov 02 12:11:01.648943 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.648857    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsh8b\" (UniqueName: \"kubernetes.io/projected/04230821-2095-474d-aa72-8dc32b08ca0f-kube-api-access-tsh8b\") pod \"payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84\" (UID: \"04230821-2095-474d-aa72-8dc32b08ca0f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84"
Nov 02 12:11:01.674640 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.674548    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-7bd72e\""
Nov 02 12:11:01.739326 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.735852    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-7bd72e-dockercfg-jpms4\""
Nov 02 12:11:01.743766 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.743730    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84"]
Nov 02 12:11:01.749978 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.749940    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsh8b\" (UniqueName: \"kubernetes.io/projected/04230821-2095-474d-aa72-8dc32b08ca0f-kube-api-access-tsh8b\") pod \"payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84\" (UID: \"04230821-2095-474d-aa72-8dc32b08ca0f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84"
Nov 02 12:11:01.750158 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.750105    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04230821-2095-474d-aa72-8dc32b08ca0f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84\" (UID: \"04230821-2095-474d-aa72-8dc32b08ca0f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84"
Nov 02 12:11:01.750901 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.750879    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04230821-2095-474d-aa72-8dc32b08ca0f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84\" (UID: \"04230821-2095-474d-aa72-8dc32b08ca0f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84"
Nov 02 12:11:01.787617 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.787588    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k"]
Nov 02 12:11:01.791721 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.791701    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k"
Nov 02 12:11:01.813250 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.813227    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsh8b\" (UniqueName: \"kubernetes.io/projected/04230821-2095-474d-aa72-8dc32b08ca0f-kube-api-access-tsh8b\") pod \"payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84\" (UID: \"04230821-2095-474d-aa72-8dc32b08ca0f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84"
Nov 02 12:11:01.817876 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.817850    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k"]
Nov 02 12:11:01.851464 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.851431    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fea24ab7-7d20-406c-9441-9fa3fdf93f61-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k\" (UID: \"fea24ab7-7d20-406c-9441-9fa3fdf93f61\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k"
Nov 02 12:11:01.851665 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.851645    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clrdf\" (UniqueName: \"kubernetes.io/projected/fea24ab7-7d20-406c-9441-9fa3fdf93f61-kube-api-access-clrdf\") pod \"payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k\" (UID: \"fea24ab7-7d20-406c-9441-9fa3fdf93f61\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k"
Nov 02 12:11:01.934381 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.934307    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84"
Nov 02 12:11:01.952344 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.952286    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clrdf\" (UniqueName: \"kubernetes.io/projected/fea24ab7-7d20-406c-9441-9fa3fdf93f61-kube-api-access-clrdf\") pod \"payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k\" (UID: \"fea24ab7-7d20-406c-9441-9fa3fdf93f61\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k"
Nov 02 12:11:01.952472 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.952378    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fea24ab7-7d20-406c-9441-9fa3fdf93f61-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k\" (UID: \"fea24ab7-7d20-406c-9441-9fa3fdf93f61\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k"
Nov 02 12:11:01.953559 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:01.953497    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fea24ab7-7d20-406c-9441-9fa3fdf93f61-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k\" (UID: \"fea24ab7-7d20-406c-9441-9fa3fdf93f61\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k"
Nov 02 12:11:02.018443 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:02.018382    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clrdf\" (UniqueName: \"kubernetes.io/projected/fea24ab7-7d20-406c-9441-9fa3fdf93f61-kube-api-access-clrdf\") pod \"payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k\" (UID: \"fea24ab7-7d20-406c-9441-9fa3fdf93f61\") " pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k"
Nov 02 12:11:02.105730 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:02.105702    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k"
Nov 02 12:11:02.320011 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:02.319980    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84"]
Nov 02 12:11:02.320608 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:11:02.320580    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04230821_2095_474d_aa72_8dc32b08ca0f.slice/crio-763836ad7be9243c48ccda494302c94d3feacb8ba547c65792969178d2eb652d WatchSource:0}: Error finding container 763836ad7be9243c48ccda494302c94d3feacb8ba547c65792969178d2eb652d: Status 404 returned error can't find the container with id 763836ad7be9243c48ccda494302c94d3feacb8ba547c65792969178d2eb652d
Nov 02 12:11:02.483539 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:02.483514    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k"]
Nov 02 12:11:02.485660 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:11:02.485634    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea24ab7_7d20_406c_9441_9fa3fdf93f61.slice/crio-a8f630d8793f44967bcde8555794d0ff36486f00f31cbea47a702c73b3ae4f14 WatchSource:0}: Error finding container a8f630d8793f44967bcde8555794d0ff36486f00f31cbea47a702c73b3ae4f14: Status 404 returned error can't find the container with id a8f630d8793f44967bcde8555794d0ff36486f00f31cbea47a702c73b3ae4f14
Nov 02 12:11:02.550141 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:02.550109    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k" event={"ID":"fea24ab7-7d20-406c-9441-9fa3fdf93f61","Type":"ContainerStarted","Data":"8766019d3f5d9ced89e908af4652ab25c2b839246e49ca097ad370ab239caa4c"}
Nov 02 12:11:02.550245 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:02.550164    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k" event={"ID":"fea24ab7-7d20-406c-9441-9fa3fdf93f61","Type":"ContainerStarted","Data":"a8f630d8793f44967bcde8555794d0ff36486f00f31cbea47a702c73b3ae4f14"}
Nov 02 12:11:02.550245 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:02.550196    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k"
Nov 02 12:11:02.552012 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:02.551985    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84" event={"ID":"04230821-2095-474d-aa72-8dc32b08ca0f","Type":"ContainerStarted","Data":"318e5e6c64043b07c9a42a7bfc4903d72a9ba9cdf0688364ef789635d9e94ac6"}
Nov 02 12:11:02.552105 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:02.552035    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84" event={"ID":"04230821-2095-474d-aa72-8dc32b08ca0f","Type":"ContainerStarted","Data":"763836ad7be9243c48ccda494302c94d3feacb8ba547c65792969178d2eb652d"}
Nov 02 12:11:02.552157 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:02.552119    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84"
Nov 02 12:11:02.666657 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:02.666612    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k" podStartSLOduration=1.666597978 podStartE2EDuration="1.666597978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:11:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:11:02.664968777 +0000 UTC m=+889.236318535" watchObservedRunningTime="2026-11-02 12:11:02.666597978 +0000 UTC m=+889.237947737"
Nov 02 12:11:02.786837 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:02.786772    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84" podStartSLOduration=1.7867533500000001 podStartE2EDuration="1.78675335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:11:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:11:02.784581892 +0000 UTC m=+889.355931649" watchObservedRunningTime="2026-11-02 12:11:02.78675335 +0000 UTC m=+889.358103106"
Nov 02 12:11:04.648447 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:04.648414    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"]
Nov 02 12:11:04.652366 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:04.652348    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:11:04.680387 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:04.680361    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt\" (UID: \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:11:04.680519 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:04.680402    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq762\" (UniqueName: \"kubernetes.io/projected/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-kube-api-access-xq762\") pod \"maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt\" (UID: \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:11:04.680595 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:04.680571    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt\" (UID: \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:11:04.680651 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:04.680614    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt\" (UID: \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:11:04.757217 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:04.757191    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w4-03c4e8-serving-cert\""
Nov 02 12:11:04.760372 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:04.760344    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"]
Nov 02 12:11:04.781892 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:04.781857    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt\" (UID: \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:11:04.782034 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:04.781926    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq762\" (UniqueName: \"kubernetes.io/projected/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-kube-api-access-xq762\") pod \"maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt\" (UID: \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:11:04.782098 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:04.782082    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt\" (UID: \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:11:04.782157 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:04.782110    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt\" (UID: \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:11:04.782320 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:11:04.782279    4942 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-serving-cert: secret "maas-api-e2e-shared-b-w4-03c4e8-serving-cert" not found
Nov 02 12:11:04.782407 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:11:04.782382    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-maas-api-tls podName:21dddcb9-f9ca-4fae-9744-1d03cbeb529d nodeName:}" failed. No retries permitted until 2026-11-02 12:11:05.282360871 +0000 UTC m=+891.853710613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-maas-api-tls") pod "maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt" (UID: "21dddcb9-f9ca-4fae-9744-1d03cbeb529d") : secret "maas-api-e2e-shared-b-w4-03c4e8-serving-cert" not found
Nov 02 12:11:04.782700 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:04.782673    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt\" (UID: \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:11:04.784673 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:04.784653    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt\" (UID: \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:11:04.843381 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:04.843350    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq762\" (UniqueName: \"kubernetes.io/projected/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-kube-api-access-xq762\") pod \"maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt\" (UID: \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:11:05.287353 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:05.287313    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt\" (UID: \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:11:05.289774 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:05.289749    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt\" (UID: \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:11:05.565964 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:05.565886    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:11:05.952232 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:05.952192    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"]
Nov 02 12:11:05.956175 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:11:05.956118    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21dddcb9_f9ca_4fae_9744_1d03cbeb529d.slice/crio-21cb87d2b60174150523a88bfdbe2922569c0dcb7e9c153a9a2a549c2c80f008 WatchSource:0}: Error finding container 21cb87d2b60174150523a88bfdbe2922569c0dcb7e9c153a9a2a549c2c80f008: Status 404 returned error can't find the container with id 21cb87d2b60174150523a88bfdbe2922569c0dcb7e9c153a9a2a549c2c80f008
Nov 02 12:11:06.575070 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:06.575033    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt" event={"ID":"21dddcb9-f9ca-4fae-9744-1d03cbeb529d","Type":"ContainerStarted","Data":"21cb87d2b60174150523a88bfdbe2922569c0dcb7e9c153a9a2a549c2c80f008"}
Nov 02 12:11:07.583380 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:07.583344    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt" event={"ID":"21dddcb9-f9ca-4fae-9744-1d03cbeb529d","Type":"ContainerStarted","Data":"2ac721eb8c3152e6acd340db8c9590220c16988bda2414fcfb6091dc411d8f2a"}
Nov 02 12:11:07.583781 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:07.583453    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:11:07.702684 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:07.702621    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt" podStartSLOduration=2.80818521 podStartE2EDuration="3.702606143s" totalImagesPullingTime="894.420933ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:11:07.700682347 +0000 UTC m=+894.272032104" watchObservedRunningTime="2026-11-02 12:11:07.702606143 +0000 UTC m=+894.273955900"
Nov 02 12:11:13.560582 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:13.560538    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84"
Nov 02 12:11:13.560959 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:13.560610    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k"
Nov 02 12:11:13.594462 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:13.594428    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:11:13.654199 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:13.654172    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:11:13.658281 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:13.658255    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:11:27.694443 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.694408    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"]
Nov 02 12:11:27.701798 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.701777    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.751101 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.751071    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-workload-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.751249 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.751203    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istiod-ca-cert\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.751322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.751249    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-envoy\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.751322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.751288    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-625s4\" (UniqueName: \"kubernetes.io/projected/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-kube-api-access-625s4\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.751469 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.751448    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-podinfo\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.751535 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.751511    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-credential-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.751580 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.751541    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-data\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.751650 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.751629    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-token\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.751701 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.751685    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-workload-certs\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.754134 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.754114    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-4b9210-openshift-default-dockercfg-fqcg6\""
Nov 02 12:11:27.806922 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.806881    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"]
Nov 02 12:11:27.853068 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.853040    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-token\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.853249 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.853094    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-workload-certs\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.853249 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.853130    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-workload-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.853249 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.853199    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istiod-ca-cert\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.853249 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.853222    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-envoy\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.853503 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.853251    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-625s4\" (UniqueName: \"kubernetes.io/projected/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-kube-api-access-625s4\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.853503 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.853309    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-podinfo\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.853503 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.853344    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-credential-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.853503 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.853372    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-data\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.853712 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.853600    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-workload-certs\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.853712 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.853638    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-workload-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.853712 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.853659    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-data\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.853948 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.853916    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-credential-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.854069 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.854013    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istiod-ca-cert\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.856220 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.856201    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-envoy\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.856478 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.856458    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-podinfo\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.913604 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.913581    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-token\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:27.913730 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:27.913669    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-625s4\" (UniqueName: \"kubernetes.io/projected/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-kube-api-access-625s4\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:28.014738 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:28.014613    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:28.456469 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:28.456442    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"]
Nov 02 12:11:28.457773 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:11:28.457751    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c2e17ba_c9fa_4b66_bed8_0d4f938cdb42.slice/crio-bc84c9b723648f848eda41585caf642633837eefc23ee8bef65440a2b99b8429 WatchSource:0}: Error finding container bc84c9b723648f848eda41585caf642633837eefc23ee8bef65440a2b99b8429: Status 404 returned error can't find the container with id bc84c9b723648f848eda41585caf642633837eefc23ee8bef65440a2b99b8429
Nov 02 12:11:28.460155 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:28.460119    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:11:28.460311 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:28.460195    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:11:28.460311 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:28.460225    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:11:28.738931 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:28.738849    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm" event={"ID":"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42","Type":"ContainerStarted","Data":"78807048a36a37184351d0d9060a28c842655c31d55b0275f8afb3c0d1562883"}
Nov 02 12:11:28.738931 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:28.738910    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm" event={"ID":"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42","Type":"ContainerStarted","Data":"bc84c9b723648f848eda41585caf642633837eefc23ee8bef65440a2b99b8429"}
Nov 02 12:11:28.861759 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:28.861702    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm" podStartSLOduration=1.8616875849999999 podStartE2EDuration="1.861687585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:11:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:11:28.858201269 +0000 UTC m=+915.429551056" watchObservedRunningTime="2026-11-02 12:11:28.861687585 +0000 UTC m=+915.433037341"
Nov 02 12:11:29.015356 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:29.015270    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:29.016961 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:29.016924    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 02 12:11:29.017100 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:29.016979    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm" podUID="7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 02 12:11:30.015307 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:30.015256    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 02 12:11:30.015796 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:30.015325    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm" podUID="7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 02 12:11:31.006894 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:11:31.006860    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:11:31.015323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:31.015283    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 02 12:11:31.015657 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:31.015351    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm" podUID="7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 02 12:11:32.015198 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:32.015164    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 02 12:11:32.015386 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:32.015221    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm" podUID="7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 02 12:11:33.015314 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:33.015276    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 02 12:11:33.015484 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:33.015358    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm" podUID="7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 02 12:11:34.015609 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:34.015571    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 02 12:11:34.015999 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:34.015635    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm" podUID="7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 02 12:11:35.015331 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.015278    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Nov 02 12:11:35.015549 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.015361    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm" podUID="7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Nov 02 12:11:35.045710 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.045677    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"]
Nov 02 12:11:35.049874 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.049857    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:11:35.101866 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.101836    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-7bd72e-serving-cert\""
Nov 02 12:11:35.147237 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.147199    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-metrics-certs\") pod \"maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x\" (UID: \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:11:35.147436 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.147261    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-maas-api-tls\") pod \"maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x\" (UID: \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:11:35.147436 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.147291    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x\" (UID: \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:11:35.147436 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.147400    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nslkz\" (UniqueName: \"kubernetes.io/projected/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-kube-api-access-nslkz\") pod \"maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x\" (UID: \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:11:35.148003 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.147980    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"]
Nov 02 12:11:35.249201 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.249154    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-metrics-certs\") pod \"maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x\" (UID: \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:11:35.249423 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.249327    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-maas-api-tls\") pod \"maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x\" (UID: \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:11:35.249423 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.249358    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x\" (UID: \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:11:35.249553 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.249432    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nslkz\" (UniqueName: \"kubernetes.io/projected/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-kube-api-access-nslkz\") pod \"maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x\" (UID: \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:11:35.249553 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:11:35.249473    4942 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-serving-cert: secret "maas-api-e2e-inf-a-7bd72e-serving-cert" not found
Nov 02 12:11:35.249553 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:11:35.249551    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-maas-api-tls podName:d6ec5343-b4bf-4508-819a-bb348b6dd0aa nodeName:}" failed. No retries permitted until 2026-11-02 12:11:35.749527529 +0000 UTC m=+922.320877279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-maas-api-tls") pod "maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x" (UID: "d6ec5343-b4bf-4508-819a-bb348b6dd0aa") : secret "maas-api-e2e-inf-a-7bd72e-serving-cert" not found
Nov 02 12:11:35.250396 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.250370    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x\" (UID: \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:11:35.251728 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.251705    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-metrics-certs\") pod \"maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x\" (UID: \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:11:35.311419 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.311351    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nslkz\" (UniqueName: \"kubernetes.io/projected/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-kube-api-access-nslkz\") pod \"maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x\" (UID: \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:11:35.756151 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.756115    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-maas-api-tls\") pod \"maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x\" (UID: \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:11:35.758954 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.758920    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-maas-api-tls\") pod \"maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x\" (UID: \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:11:35.970413 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:35.970377    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:11:36.026415 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:36.022511    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:36.026415 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:36.024243    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:36.026415 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:36.024901    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:11:36.414414 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:36.414337    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"]
Nov 02 12:11:36.417502 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:11:36.417470    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6ec5343_b4bf_4508_819a_bb348b6dd0aa.slice/crio-2c149a97d8020782501c5b20afb8d7cda1a683444f24cd4e78f660894ffb6478 WatchSource:0}: Error finding container 2c149a97d8020782501c5b20afb8d7cda1a683444f24cd4e78f660894ffb6478: Status 404 returned error can't find the container with id 2c149a97d8020782501c5b20afb8d7cda1a683444f24cd4e78f660894ffb6478
Nov 02 12:11:36.814745 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:36.814646    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x" event={"ID":"d6ec5343-b4bf-4508-819a-bb348b6dd0aa","Type":"ContainerStarted","Data":"2c149a97d8020782501c5b20afb8d7cda1a683444f24cd4e78f660894ffb6478"}
Nov 02 12:11:37.822517 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:37.822482    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x" event={"ID":"d6ec5343-b4bf-4508-819a-bb348b6dd0aa","Type":"ContainerStarted","Data":"1b0c2246f490b639dc3a118fa150545dfdce0a3833350edd128d03441f1a1309"}
Nov 02 12:11:37.822997 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:37.822558    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:11:37.942191 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:37.942133    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x" podStartSLOduration=2.10677249 podStartE2EDuration="2.942119722s" totalImagesPullingTime="835.347232ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:11:37.939403274 +0000 UTC m=+924.510753056" watchObservedRunningTime="2026-11-02 12:11:37.942119722 +0000 UTC m=+924.513469479"
Nov 02 12:11:43.835169 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:11:43.835128    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:12:00.983335 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:12:00.983223    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:12:09.365344 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.365312    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"]
Nov 02 12:12:09.370473 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.370454    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.427913 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.427877    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"]
Nov 02 12:12:09.448634 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.448598    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-home\") pod \"auth-test-model-dd087a-kserve-8df5d595c-bgnk8\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.448851 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.448742    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-model-cache\") pod \"auth-test-model-dd087a-kserve-8df5d595c-bgnk8\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.448851 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.448789    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-tls-certs\") pod \"auth-test-model-dd087a-kserve-8df5d595c-bgnk8\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.448961 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.448881    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-dshm\") pod \"auth-test-model-dd087a-kserve-8df5d595c-bgnk8\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.448961 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.448919    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-tmp-dir\") pod \"auth-test-model-dd087a-kserve-8df5d595c-bgnk8\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.449074 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.449052    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckd8m\" (UniqueName: \"kubernetes.io/projected/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-kube-api-access-ckd8m\") pod \"auth-test-model-dd087a-kserve-8df5d595c-bgnk8\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.474009 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.473987    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w4-dd087a\"/\"kube-root-ca.crt\""
Nov 02 12:12:09.474009 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.473999    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w4-dd087a\"/\"openshift-service-ca.crt\""
Nov 02 12:12:09.474196 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.474010    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w4-dd087a\"/\"default-dockercfg-s8lc4\""
Nov 02 12:12:09.474196 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.474073    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w4-dd087a\"/\"auth-test-model-dd087a-kserve-self-signed-certs\""
Nov 02 12:12:09.550007 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.549976    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-home\") pod \"auth-test-model-dd087a-kserve-8df5d595c-bgnk8\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.550183 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.550031    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-model-cache\") pod \"auth-test-model-dd087a-kserve-8df5d595c-bgnk8\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.550183 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.550059    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-tls-certs\") pod \"auth-test-model-dd087a-kserve-8df5d595c-bgnk8\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.550183 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.550081    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-dshm\") pod \"auth-test-model-dd087a-kserve-8df5d595c-bgnk8\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.550183 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.550099    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-tmp-dir\") pod \"auth-test-model-dd087a-kserve-8df5d595c-bgnk8\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.550183 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.550161    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckd8m\" (UniqueName: \"kubernetes.io/projected/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-kube-api-access-ckd8m\") pod \"auth-test-model-dd087a-kserve-8df5d595c-bgnk8\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.550515 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.550403    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-home\") pod \"auth-test-model-dd087a-kserve-8df5d595c-bgnk8\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.550515 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.550499    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-model-cache\") pod \"auth-test-model-dd087a-kserve-8df5d595c-bgnk8\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.550599 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.550555    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-tmp-dir\") pod \"auth-test-model-dd087a-kserve-8df5d595c-bgnk8\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.552457 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.552434    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-dshm\") pod \"auth-test-model-dd087a-kserve-8df5d595c-bgnk8\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.552717 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.552699    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-tls-certs\") pod \"auth-test-model-dd087a-kserve-8df5d595c-bgnk8\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.613662 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.613628    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckd8m\" (UniqueName: \"kubernetes.io/projected/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-kube-api-access-ckd8m\") pod \"auth-test-model-dd087a-kserve-8df5d595c-bgnk8\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:09.682980 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:09.682910    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:10.059704 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:10.059672    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"]
Nov 02 12:12:10.061453 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:12:10.061425    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8d2041b_9e25_42f5_aee6_8e4ac3a441d2.slice/crio-256f58f76e237e9f6b187d86629685afb374ec68e5f84cf980a7885f46180b1c WatchSource:0}: Error finding container 256f58f76e237e9f6b187d86629685afb374ec68e5f84cf980a7885f46180b1c: Status 404 returned error can't find the container with id 256f58f76e237e9f6b187d86629685afb374ec68e5f84cf980a7885f46180b1c
Nov 02 12:12:11.057262 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:11.057223    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8" event={"ID":"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2","Type":"ContainerStarted","Data":"4caa2503921977a916800847bda03bac368ff45b776f391600879767bbb31990"}
Nov 02 12:12:11.057675 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:11.057272    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8" event={"ID":"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2","Type":"ContainerStarted","Data":"256f58f76e237e9f6b187d86629685afb374ec68e5f84cf980a7885f46180b1c"}
Nov 02 12:12:11.175818 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:11.175763    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8" podStartSLOduration=2.175748733 podStartE2EDuration="2.175748733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:12:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:12:11.173395606 +0000 UTC m=+957.744745364" watchObservedRunningTime="2026-11-02 12:12:11.175748733 +0000 UTC m=+957.747098520"
Nov 02 12:12:11.683354 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:11.683325    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:13.039846 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.039811    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"]
Nov 02 12:12:13.043837 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.043813    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.148938 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.148907    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"]
Nov 02 12:12:13.193809 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.193773    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-workload-certs\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.193998 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.193855    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-data\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.193998 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.193911    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-token\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.193998 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.193937    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sn8r\" (UniqueName: \"kubernetes.io/projected/feca8edf-3c6e-4370-8e3c-c68e195c655c-kube-api-access-5sn8r\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.193998 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.193980    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-envoy\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.194214 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.194073    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-workload-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.194214 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.194127    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-credential-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.194351 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.194235    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-podinfo\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.194351 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.194264    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/feca8edf-3c6e-4370-8e3c-c68e195c655c-istiod-ca-cert\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.295188 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.295119    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-token\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.295188 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.295153    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sn8r\" (UniqueName: \"kubernetes.io/projected/feca8edf-3c6e-4370-8e3c-c68e195c655c-kube-api-access-5sn8r\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.295188 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.295187    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-envoy\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.295409 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.295250    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-workload-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.295409 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.295321    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-credential-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.295409 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.295395    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-podinfo\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.295564 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.295414    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/feca8edf-3c6e-4370-8e3c-c68e195c655c-istiod-ca-cert\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.295564 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.295461    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-workload-certs\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.295564 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.295514    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-data\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.295796 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.295768    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-workload-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.295877 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.295796    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-credential-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.295877 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.295865    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-data\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.295989 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.295944    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-workload-certs\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.296122 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.296104    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/feca8edf-3c6e-4370-8e3c-c68e195c655c-istiod-ca-cert\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.297487 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.297468    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-envoy\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.297768 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.297752    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-podinfo\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.355684 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.355657    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-token\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.355888 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.355862    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sn8r\" (UniqueName: \"kubernetes.io/projected/feca8edf-3c6e-4370-8e3c-c68e195c655c-kube-api-access-5sn8r\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.359515 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.359490    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:13.794865 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.794839    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"]
Nov 02 12:12:13.796067 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:12:13.796040    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeca8edf_3c6e_4370_8e3c_c68e195c655c.slice/crio-d93a4f7c04741b69883e3740ead6dbcfe8666ea4040bda090b46c1818fdbaf2a WatchSource:0}: Error finding container d93a4f7c04741b69883e3740ead6dbcfe8666ea4040bda090b46c1818fdbaf2a: Status 404 returned error can't find the container with id d93a4f7c04741b69883e3740ead6dbcfe8666ea4040bda090b46c1818fdbaf2a
Nov 02 12:12:13.798690 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.798640    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:12:13.798789 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.798724    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:12:13.798789 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:13.798767    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:12:14.079434 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:14.079335    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m" event={"ID":"feca8edf-3c6e-4370-8e3c-c68e195c655c","Type":"ContainerStarted","Data":"9a00fff8534f3c669c450142d6a88b1eae803bb39131ac0bbf5ec126a3a471fb"}
Nov 02 12:12:14.079434 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:14.079401    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m" event={"ID":"feca8edf-3c6e-4370-8e3c-c68e195c655c","Type":"ContainerStarted","Data":"d93a4f7c04741b69883e3740ead6dbcfe8666ea4040bda090b46c1818fdbaf2a"}
Nov 02 12:12:14.203556 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:14.203498    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m" podStartSLOduration=2.203483682 podStartE2EDuration="2.203483682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:12:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:12:14.200042225 +0000 UTC m=+960.771391984" watchObservedRunningTime="2026-11-02 12:12:14.203483682 +0000 UTC m=+960.774833439"
Nov 02 12:12:14.364880 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:14.364847    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:14.365246 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:14.365222    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 12:12:14.365433 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:14.365407    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m" podUID="feca8edf-3c6e-4370-8e3c-c68e195c655c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 12:12:15.373749 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:15.373709    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 12:12:15.374351 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:15.374291    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m" podUID="feca8edf-3c6e-4370-8e3c-c68e195c655c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 12:12:16.367322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:16.366693    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 12:12:16.367322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:16.366756    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m" podUID="feca8edf-3c6e-4370-8e3c-c68e195c655c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 12:12:17.364322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:17.361708    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 12:12:17.364322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:17.361769    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m" podUID="feca8edf-3c6e-4370-8e3c-c68e195c655c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 12:12:18.375329 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:18.369754    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 12:12:18.375329 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:18.369838    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m" podUID="feca8edf-3c6e-4370-8e3c-c68e195c655c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 12:12:19.370719 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:19.370680    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 12:12:19.370994 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:19.370965    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m" podUID="feca8edf-3c6e-4370-8e3c-c68e195c655c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 12:12:19.683527 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:19.683439    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:19.696876 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:19.696845    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:20.131746 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:20.131716    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:12:20.360112 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:20.360076    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 02 12:12:20.360266 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:20.360148    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m" podUID="feca8edf-3c6e-4370-8e3c-c68e195c655c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 02 12:12:21.642699 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:21.642663    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:21.643175 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:21.643084    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:21.644099 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:21.644076    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:12:30.982969 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:12:30.982932    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:12:49.408156 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:49.408122    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn"]
Nov 02 12:12:54.409171 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:54.409129    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn" podUID="cb8c82c6-339a-4880-a890-2b3effe841ae" containerName="payload-processing" containerID="cri-o://452d921257b2193dd2757b1dd10e991fa5b984ca2500660078cc6a91e44f4835" gracePeriod=25
Nov 02 12:12:57.433325 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:57.433252    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn" podUID="cb8c82c6-339a-4880-a890-2b3effe841ae" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 12:12:58.411008 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.410976    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"]
Nov 02 12:12:58.416492 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.416472    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.445014 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.444989    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-workload-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.445361 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.445033    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-workload-certs\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.445361 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.445113    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-credential-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.445361 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.445164    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-istio-envoy\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.445361 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.445193    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnnfh\" (UniqueName: \"kubernetes.io/projected/efd23a38-957d-44b6-b345-e038bc4549bb-kube-api-access-bnnfh\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.445361 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.445262    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efd23a38-957d-44b6-b345-e038bc4549bb-istio-podinfo\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.445361 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.445292    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efd23a38-957d-44b6-b345-e038bc4549bb-istio-token\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.445590 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.445402    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-istio-data\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.445590 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.445419    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efd23a38-957d-44b6-b345-e038bc4549bb-istiod-ca-cert\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.510610 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.510572    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"]
Nov 02 12:12:58.546060 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.546030    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-workload-certs\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.546213 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.546090    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-credential-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.546213 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.546125    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-istio-envoy\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.546213 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.546166    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnnfh\" (UniqueName: \"kubernetes.io/projected/efd23a38-957d-44b6-b345-e038bc4549bb-kube-api-access-bnnfh\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.546213 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.546210    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efd23a38-957d-44b6-b345-e038bc4549bb-istio-podinfo\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.546464 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.546240    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efd23a38-957d-44b6-b345-e038bc4549bb-istio-token\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.546464 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.546372    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-istio-data\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.546464 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.546401    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efd23a38-957d-44b6-b345-e038bc4549bb-istiod-ca-cert\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.546624 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.546485    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-workload-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.546624 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.546509    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-workload-certs\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.546624 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.546593    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-credential-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.546808 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.546785    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-istio-data\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.546866 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.546848    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-workload-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.547139 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.547122    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efd23a38-957d-44b6-b345-e038bc4549bb-istiod-ca-cert\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.548494 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.548472    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-istio-envoy\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.548763 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.548742    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efd23a38-957d-44b6-b345-e038bc4549bb-istio-podinfo\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.606892 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.606861    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efd23a38-957d-44b6-b345-e038bc4549bb-istio-token\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.607374 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.607347    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnnfh\" (UniqueName: \"kubernetes.io/projected/efd23a38-957d-44b6-b345-e038bc4549bb-kube-api-access-bnnfh\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:58.730507 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:58.730429    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:12:59.162494 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:59.162463    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"]
Nov 02 12:12:59.163503 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:12:59.163473    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefd23a38_957d_44b6_b345_e038bc4549bb.slice/crio-6bc0ccdf28c1f6cdd39d62de38123849b899d1986d60e93437caf359856fb9c8 WatchSource:0}: Error finding container 6bc0ccdf28c1f6cdd39d62de38123849b899d1986d60e93437caf359856fb9c8: Status 404 returned error can't find the container with id 6bc0ccdf28c1f6cdd39d62de38123849b899d1986d60e93437caf359856fb9c8
Nov 02 12:12:59.166085 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:59.166051    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:12:59.166161 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:59.166124    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:12:59.166161 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:59.166152    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:12:59.385139 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:59.385094    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22" event={"ID":"efd23a38-957d-44b6-b345-e038bc4549bb","Type":"ContainerStarted","Data":"09e46f286e313066d6e39fdcde7fd55679ae58d5a7b06bb7584b2f0440372619"}
Nov 02 12:12:59.385362 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:59.385165    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22" event={"ID":"efd23a38-957d-44b6-b345-e038bc4549bb","Type":"ContainerStarted","Data":"6bc0ccdf28c1f6cdd39d62de38123849b899d1986d60e93437caf359856fb9c8"}
Nov 02 12:12:59.506778 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:59.506675    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22" podStartSLOduration=1.50665541 podStartE2EDuration="1.50665541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:12:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:12:59.50527713 +0000 UTC m=+1006.076626889" watchObservedRunningTime="2026-11-02 12:12:59.50665541 +0000 UTC m=+1006.078005169"
Nov 02 12:12:59.730990 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:12:59.730962    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:13:00.732324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:00.732282    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 02 12:13:00.732698 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:00.732343    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22" podUID="efd23a38-957d-44b6-b345-e038bc4549bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 02 12:13:01.004635 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:13:01.004536    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:13:01.731051 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:01.731015    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 02 12:13:01.731228 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:01.731070    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22" podUID="efd23a38-957d-44b6-b345-e038bc4549bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 02 12:13:02.731360 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:02.731324    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 02 12:13:02.731747 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:02.731378    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22" podUID="efd23a38-957d-44b6-b345-e038bc4549bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 02 12:13:03.731538 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:03.731500    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 02 12:13:03.731950 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:03.731556    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22" podUID="efd23a38-957d-44b6-b345-e038bc4549bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 02 12:13:04.731493 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:04.731453    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 02 12:13:04.731663 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:04.731512    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22" podUID="efd23a38-957d-44b6-b345-e038bc4549bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 02 12:13:06.098455 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:06.098416    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:13:06.099247 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:06.099225    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:13:06.100140 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:06.100114    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:13:06.649844 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:06.649813    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"]
Nov 02 12:13:06.653880 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:06.653858    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:13:06.710376 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:06.710354    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-036f88-serving-cert\""
Nov 02 12:13:06.713933 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:06.713909    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"]
Nov 02 12:13:06.753999 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:06.753969    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4r92\" (UniqueName: \"kubernetes.io/projected/958b6369-5a4b-4a87-832e-66bc5e2db790-kube-api-access-z4r92\") pod \"maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f\" (UID: \"958b6369-5a4b-4a87-832e-66bc5e2db790\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:13:06.754131 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:06.754016    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/958b6369-5a4b-4a87-832e-66bc5e2db790-maas-api-tls\") pod \"maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f\" (UID: \"958b6369-5a4b-4a87-832e-66bc5e2db790\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:13:06.754202 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:06.754143    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/958b6369-5a4b-4a87-832e-66bc5e2db790-metrics-certs\") pod \"maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f\" (UID: \"958b6369-5a4b-4a87-832e-66bc5e2db790\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:13:06.754342 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:06.754324    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/958b6369-5a4b-4a87-832e-66bc5e2db790-openshift-service-ca\") pod \"maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f\" (UID: \"958b6369-5a4b-4a87-832e-66bc5e2db790\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:13:06.855198 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:06.855167    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/958b6369-5a4b-4a87-832e-66bc5e2db790-maas-api-tls\") pod \"maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f\" (UID: \"958b6369-5a4b-4a87-832e-66bc5e2db790\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:13:06.855395 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:06.855244    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/958b6369-5a4b-4a87-832e-66bc5e2db790-metrics-certs\") pod \"maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f\" (UID: \"958b6369-5a4b-4a87-832e-66bc5e2db790\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:13:06.855395 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:13:06.855338    4942 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-serving-cert: secret "maas-api-e2e-inf-b-036f88-serving-cert" not found
Nov 02 12:13:06.855515 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:06.855391    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/958b6369-5a4b-4a87-832e-66bc5e2db790-openshift-service-ca\") pod \"maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f\" (UID: \"958b6369-5a4b-4a87-832e-66bc5e2db790\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:13:06.855515 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:13:06.855408    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/958b6369-5a4b-4a87-832e-66bc5e2db790-maas-api-tls podName:958b6369-5a4b-4a87-832e-66bc5e2db790 nodeName:}" failed. No retries permitted until 2026-11-02 12:13:07.355389399 +0000 UTC m=+1013.926739134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/958b6369-5a4b-4a87-832e-66bc5e2db790-maas-api-tls") pod "maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f" (UID: "958b6369-5a4b-4a87-832e-66bc5e2db790") : secret "maas-api-e2e-inf-b-036f88-serving-cert" not found
Nov 02 12:13:06.855515 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:06.855479    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4r92\" (UniqueName: \"kubernetes.io/projected/958b6369-5a4b-4a87-832e-66bc5e2db790-kube-api-access-z4r92\") pod \"maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f\" (UID: \"958b6369-5a4b-4a87-832e-66bc5e2db790\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:13:06.856090 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:06.856065    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/958b6369-5a4b-4a87-832e-66bc5e2db790-openshift-service-ca\") pod \"maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f\" (UID: \"958b6369-5a4b-4a87-832e-66bc5e2db790\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:13:06.857992 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:06.857966    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/958b6369-5a4b-4a87-832e-66bc5e2db790-metrics-certs\") pod \"maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f\" (UID: \"958b6369-5a4b-4a87-832e-66bc5e2db790\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:13:06.917153 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:06.917084    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4r92\" (UniqueName: \"kubernetes.io/projected/958b6369-5a4b-4a87-832e-66bc5e2db790-kube-api-access-z4r92\") pod \"maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f\" (UID: \"958b6369-5a4b-4a87-832e-66bc5e2db790\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:13:07.360509 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:07.360476    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/958b6369-5a4b-4a87-832e-66bc5e2db790-maas-api-tls\") pod \"maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f\" (UID: \"958b6369-5a4b-4a87-832e-66bc5e2db790\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:13:07.363006 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:07.362975    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/958b6369-5a4b-4a87-832e-66bc5e2db790-maas-api-tls\") pod \"maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f\" (UID: \"958b6369-5a4b-4a87-832e-66bc5e2db790\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:13:07.434065 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:07.434032    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn" podUID="cb8c82c6-339a-4880-a890-2b3effe841ae" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 12:13:07.567514 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:07.567487    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:13:07.944620 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:07.944595    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"]
Nov 02 12:13:07.947053 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:13:07.947026    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod958b6369_5a4b_4a87_832e_66bc5e2db790.slice/crio-97afdab67f3b56cb8d59beb8cc8b6d92ce32531c562942d1a77ce811ac3b0bbb WatchSource:0}: Error finding container 97afdab67f3b56cb8d59beb8cc8b6d92ce32531c562942d1a77ce811ac3b0bbb: Status 404 returned error can't find the container with id 97afdab67f3b56cb8d59beb8cc8b6d92ce32531c562942d1a77ce811ac3b0bbb
Nov 02 12:13:07.948430 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:07.948411    4942 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 12:13:08.449922 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:08.449886    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f" event={"ID":"958b6369-5a4b-4a87-832e-66bc5e2db790","Type":"ContainerStarted","Data":"97afdab67f3b56cb8d59beb8cc8b6d92ce32531c562942d1a77ce811ac3b0bbb"}
Nov 02 12:13:09.457357 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:09.457323    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f" event={"ID":"958b6369-5a4b-4a87-832e-66bc5e2db790","Type":"ContainerStarted","Data":"03193fdb5c613ba8f9571a7ee0f99b2e14edb7c0a6ac7daed8e8edb426d4d515"}
Nov 02 12:13:09.457737 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:09.457395    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:13:09.576625 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:09.576569    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f" podStartSLOduration=2.570355455 podStartE2EDuration="3.576555301s" totalImagesPullingTime="1.006199846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:13:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:13:09.574872547 +0000 UTC m=+1016.146222305" watchObservedRunningTime="2026-11-02 12:13:09.576555301 +0000 UTC m=+1016.147905057"
Nov 02 12:13:14.679211 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:14.679156    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn"
Nov 02 12:13:14.762147 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:14.762106    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb8c82c6-339a-4880-a890-2b3effe841ae-kube-api-access-c9rw6\" (UniqueName: \"kubernetes.io/projected/cb8c82c6-339a-4880-a890-2b3effe841ae-kube-api-access-c9rw6\") pod \"cb8c82c6-339a-4880-a890-2b3effe841ae\" (UID: \"cb8c82c6-339a-4880-a890-2b3effe841ae\") "
Nov 02 12:13:14.762383 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:14.762356    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cb8c82c6-339a-4880-a890-2b3effe841ae-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb8c82c6-339a-4880-a890-2b3effe841ae-plugins-config-volume\") pod \"cb8c82c6-339a-4880-a890-2b3effe841ae\" (UID: \"cb8c82c6-339a-4880-a890-2b3effe841ae\") "
Nov 02 12:13:14.762751 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:14.762722    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb8c82c6-339a-4880-a890-2b3effe841ae-plugins-config-volume" pod "cb8c82c6-339a-4880-a890-2b3effe841ae" (UID: "cb8c82c6-339a-4880-a890-2b3effe841ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:13:14.762910 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:14.762889    4942 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb8c82c6-339a-4880-a890-2b3effe841ae-plugins-config-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:13:14.765112 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:14.765082    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb8c82c6-339a-4880-a890-2b3effe841ae-kube-api-access-c9rw6" pod "cb8c82c6-339a-4880-a890-2b3effe841ae" (UID: "cb8c82c6-339a-4880-a890-2b3effe841ae"). InnerVolumeSpecName "kube-api-access-c9rw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:13:14.863902 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:14.863870    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9rw6\" (UniqueName: \"kubernetes.io/projected/cb8c82c6-339a-4880-a890-2b3effe841ae-kube-api-access-c9rw6\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:13:15.470719 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:15.470687    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:13:15.525244 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:15.525210    4942 generic.go:356] "Generic (PLEG): container finished" podID="cb8c82c6-339a-4880-a890-2b3effe841ae" containerID="452d921257b2193dd2757b1dd10e991fa5b984ca2500660078cc6a91e44f4835" exitCode=0
Nov 02 12:13:15.525441 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:15.525279    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn"
Nov 02 12:13:15.525441 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:15.525293    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn" event={"ID":"cb8c82c6-339a-4880-a890-2b3effe841ae","Type":"ContainerDied","Data":"452d921257b2193dd2757b1dd10e991fa5b984ca2500660078cc6a91e44f4835"}
Nov 02 12:13:15.525441 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:15.525377    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn" event={"ID":"cb8c82c6-339a-4880-a890-2b3effe841ae","Type":"ContainerDied","Data":"c70a6dd8c0933e7ae9776fbb1541f14d566a3249582a735e56aed5e47246cfb0"}
Nov 02 12:13:15.525441 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:15.525405    4942 scope.go:122] "RemoveContainer" containerID="452d921257b2193dd2757b1dd10e991fa5b984ca2500660078cc6a91e44f4835"
Nov 02 12:13:15.536322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:15.536082    4942 scope.go:122] "RemoveContainer" containerID="452d921257b2193dd2757b1dd10e991fa5b984ca2500660078cc6a91e44f4835"
Nov 02 12:13:15.536468 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:13:15.536445    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"452d921257b2193dd2757b1dd10e991fa5b984ca2500660078cc6a91e44f4835\": container with ID starting with 452d921257b2193dd2757b1dd10e991fa5b984ca2500660078cc6a91e44f4835 not found: ID does not exist" containerID="452d921257b2193dd2757b1dd10e991fa5b984ca2500660078cc6a91e44f4835"
Nov 02 12:13:15.536507 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:15.536477    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"452d921257b2193dd2757b1dd10e991fa5b984ca2500660078cc6a91e44f4835"} err="failed to get container status \"452d921257b2193dd2757b1dd10e991fa5b984ca2500660078cc6a91e44f4835\": rpc error: code = NotFound desc = could not find container \"452d921257b2193dd2757b1dd10e991fa5b984ca2500660078cc6a91e44f4835\": container with ID starting with 452d921257b2193dd2757b1dd10e991fa5b984ca2500660078cc6a91e44f4835 not found: ID does not exist"
Nov 02 12:13:15.707710 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:15.707667    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn"]
Nov 02 12:13:15.710635 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:15.710612    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-b6f3d8-674788f46f-bvqgn"]
Nov 02 12:13:17.543157 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:17.543125    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cb8c82c6-339a-4880-a890-2b3effe841ae" path="/var/lib/kubelet/pods/cb8c82c6-339a-4880-a890-2b3effe841ae/volumes"
Nov 02 12:13:30.987707 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:13:30.987622    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:13:59.188970 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.188934    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"]
Nov 02 12:13:59.209747 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.209714    4942 cpu_manager.go:414] "containerMap: removing container" podUID="cb8c82c6-339a-4880-a890-2b3effe841ae" containerName="payload-processing"
Nov 02 12:13:59.209970 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.209953    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb8c82c6-339a-4880-a890-2b3effe841ae" containerName="payload-processing"
Nov 02 12:13:59.215596 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.215566    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb8c82c6-339a-4880-a890-2b3effe841ae" containerName="payload-processing"
Nov 02 12:13:59.230333 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.221595    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.300244 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.300204    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"]
Nov 02 12:13:59.309906 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.309875    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-podinfo\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.310090 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.309920    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-workload-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.310090 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.309942    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-envoy\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.310090 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.310019    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6cwk\" (UniqueName: \"kubernetes.io/projected/1c3ecabc-d682-4784-93fa-765acb5555a4-kube-api-access-d6cwk\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.310276 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.310089    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-credential-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.310276 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.310115    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-token\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.310276 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.310137    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-workload-certs\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.310276 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.310247    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c3ecabc-d682-4784-93fa-765acb5555a4-istiod-ca-cert\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.310517 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.310293    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-data\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.411859 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.411798    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c3ecabc-d682-4784-93fa-765acb5555a4-istiod-ca-cert\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.412045 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.411916    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-data\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.412045 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.411998    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-podinfo\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.412178 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.412042    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-workload-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.412178 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.412090    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-envoy\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.412178 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.412126    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6cwk\" (UniqueName: \"kubernetes.io/projected/1c3ecabc-d682-4784-93fa-765acb5555a4-kube-api-access-d6cwk\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.412363 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.412239    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-credential-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.412363 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.412279    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-token\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.412363 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.412325    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-workload-certs\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.412544 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.412360    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-data\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.412677 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.412654    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c3ecabc-d682-4784-93fa-765acb5555a4-istiod-ca-cert\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.412795 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.412656    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-workload-certs\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.413130 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.413102    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-credential-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.413505 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.413480    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-workload-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.415031 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.415004    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-envoy\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.415527 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.415501    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-podinfo\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.473780 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.473707    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-token\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.473981 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.473897    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6cwk\" (UniqueName: \"kubernetes.io/projected/1c3ecabc-d682-4784-93fa-765acb5555a4-kube-api-access-d6cwk\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:13:59.559856 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:13:59.559825    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:14:00.273739 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:00.273705    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"]
Nov 02 12:14:00.275660 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:00.275619    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:14:00.275767 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:00.275688    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:14:00.275767 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:00.275718    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:14:00.910320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:00.899113    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s" event={"ID":"1c3ecabc-d682-4784-93fa-765acb5555a4","Type":"ContainerStarted","Data":"05f523f1951c009e4026f4897c68c42842c7d23f073b8d36bc0e31dacc61df44"}
Nov 02 12:14:00.910320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:00.899186    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s" event={"ID":"1c3ecabc-d682-4784-93fa-765acb5555a4","Type":"ContainerStarted","Data":"79b0030e198b02babd004e9b4b8c5672151d2a3551d1ea3feac4e00db6f7fc51"}
Nov 02 12:14:01.100139 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:14:01.100098    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:14:01.573341 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:01.563101    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 02 12:14:01.573341 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:01.563163    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s" podUID="1c3ecabc-d682-4784-93fa-765acb5555a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 02 12:14:01.590426 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:01.590352    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:14:02.568321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:02.567661    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 02 12:14:02.568321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:02.567736    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s" podUID="1c3ecabc-d682-4784-93fa-765acb5555a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 02 12:14:03.570324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:03.562175    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 02 12:14:03.570324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:03.562233    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s" podUID="1c3ecabc-d682-4784-93fa-765acb5555a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 02 12:14:04.564718 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:04.564682    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 02 12:14:04.565025 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:04.564991    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s" podUID="1c3ecabc-d682-4784-93fa-765acb5555a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 02 12:14:05.577719 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:05.577679    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 02 12:14:05.578154 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:05.577742    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s" podUID="1c3ecabc-d682-4784-93fa-765acb5555a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 02 12:14:06.568324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:06.567684    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 02 12:14:06.568324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:06.567752    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s" podUID="1c3ecabc-d682-4784-93fa-765acb5555a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 02 12:14:07.885721 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:07.885691    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:14:07.886617 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:07.886595    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:14:07.887577 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:07.887553    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:14:08.106477 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:08.106420    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s" podStartSLOduration=9.106400901 podStartE2EDuration="9.106400901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:13:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:14:01.100956119 +0000 UTC m=+1067.672305872" watchObservedRunningTime="2026-11-02 12:14:08.106400901 +0000 UTC m=+1074.677750662"
Nov 02 12:14:16.422755 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:16.422715    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"]
Nov 02 12:14:16.423227 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:16.422974    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg" podUID="3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b" containerName="maas-api" containerID="cri-o://91b4658e66a7722ac9f12144f4f71be92234a7bddf4ba171b5fda5f1db17d853" gracePeriod=30
Nov 02 12:14:16.728011 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:16.727984    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:14:16.768464 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:16.768436    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-openshift-service-ca\") pod \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\" (UID: \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\") "
Nov 02 12:14:16.768629 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:16.768494    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-maas-api-tls\") pod \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\" (UID: \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\") "
Nov 02 12:14:16.768629 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:16.768515    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-metrics-certs\") pod \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\" (UID: \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\") "
Nov 02 12:14:16.768755 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:16.768676    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-kube-api-access-mxp5s\" (UniqueName: \"kubernetes.io/projected/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-kube-api-access-mxp5s\") pod \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\" (UID: \"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b\") "
Nov 02 12:14:16.768913 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:16.768874    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-openshift-service-ca" pod "3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b" (UID: "3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:14:16.769154 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:16.769127    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:16.770852 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:16.770823    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-kube-api-access-mxp5s" pod "3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b" (UID: "3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b"). InnerVolumeSpecName "kube-api-access-mxp5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:14:16.770961 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:16.770878    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-maas-api-tls" pod "3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b" (UID: "3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:14:16.771123 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:16.771094    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-metrics-certs" pod "3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b" (UID: "3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:14:16.871175 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:16.871139    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxp5s\" (UniqueName: \"kubernetes.io/projected/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-kube-api-access-mxp5s\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:16.871175 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:16.871174    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:16.871436 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:16.871189    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:17.057767 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:17.057685    4942 generic.go:356] "Generic (PLEG): container finished" podID="3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b" containerID="91b4658e66a7722ac9f12144f4f71be92234a7bddf4ba171b5fda5f1db17d853" exitCode=0
Nov 02 12:14:17.057767 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:17.057741    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"
Nov 02 12:14:17.057767 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:17.057757    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg" event={"ID":"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b","Type":"ContainerDied","Data":"91b4658e66a7722ac9f12144f4f71be92234a7bddf4ba171b5fda5f1db17d853"}
Nov 02 12:14:17.058118 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:17.057802    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg" event={"ID":"3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b","Type":"ContainerDied","Data":"3efce5e5682590de5d9081c61625a25a7017facd356efd584a7ddff45d5505a9"}
Nov 02 12:14:17.058118 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:17.057823    4942 scope.go:122] "RemoveContainer" containerID="91b4658e66a7722ac9f12144f4f71be92234a7bddf4ba171b5fda5f1db17d853"
Nov 02 12:14:17.068840 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:17.068818    4942 scope.go:122] "RemoveContainer" containerID="91b4658e66a7722ac9f12144f4f71be92234a7bddf4ba171b5fda5f1db17d853"
Nov 02 12:14:17.069135 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:14:17.069114    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91b4658e66a7722ac9f12144f4f71be92234a7bddf4ba171b5fda5f1db17d853\": container with ID starting with 91b4658e66a7722ac9f12144f4f71be92234a7bddf4ba171b5fda5f1db17d853 not found: ID does not exist" containerID="91b4658e66a7722ac9f12144f4f71be92234a7bddf4ba171b5fda5f1db17d853"
Nov 02 12:14:17.069198 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:17.069147    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91b4658e66a7722ac9f12144f4f71be92234a7bddf4ba171b5fda5f1db17d853"} err="failed to get container status \"91b4658e66a7722ac9f12144f4f71be92234a7bddf4ba171b5fda5f1db17d853\": rpc error: code = NotFound desc = could not find container \"91b4658e66a7722ac9f12144f4f71be92234a7bddf4ba171b5fda5f1db17d853\": container with ID starting with 91b4658e66a7722ac9f12144f4f71be92234a7bddf4ba171b5fda5f1db17d853 not found: ID does not exist"
Nov 02 12:14:17.229286 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:17.229250    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"]
Nov 02 12:14:17.234453 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:17.234429    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-b6f3d8-848b5fdd4f-kk9pg"]
Nov 02 12:14:17.542652 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:17.542621    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b" path="/var/lib/kubelet/pods/3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b/volumes"
Nov 02 12:14:31.037727 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:14:31.037672    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:14:37.868084 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:37.868046    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"]
Nov 02 12:14:37.868636 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:37.868239    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw" podUID="6712798b-0759-43a3-a5c6-a6c475ebc228" containerName="maas-api" containerID="cri-o://b12ceae0484ca2294cf2e91ef6f9da27cabde6889f78a2f317da79b385bbe263" gracePeriod=30
Nov 02 12:14:37.987103 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:37.987071    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw" podUID="6712798b-0759-43a3-a5c6-a6c475ebc228" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.63:8443/health\": dial tcp 10.132.0.63:8443: connect: connection refused"
Nov 02 12:14:38.159062 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.159004    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:14:38.239363 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.239324    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6712798b-0759-43a3-a5c6-a6c475ebc228-kube-api-access-7h72x\" (UniqueName: \"kubernetes.io/projected/6712798b-0759-43a3-a5c6-a6c475ebc228-kube-api-access-7h72x\") pod \"6712798b-0759-43a3-a5c6-a6c475ebc228\" (UID: \"6712798b-0759-43a3-a5c6-a6c475ebc228\") "
Nov 02 12:14:38.239560 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.239415    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6712798b-0759-43a3-a5c6-a6c475ebc228-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6712798b-0759-43a3-a5c6-a6c475ebc228-metrics-certs\") pod \"6712798b-0759-43a3-a5c6-a6c475ebc228\" (UID: \"6712798b-0759-43a3-a5c6-a6c475ebc228\") "
Nov 02 12:14:38.239560 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.239459    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6712798b-0759-43a3-a5c6-a6c475ebc228-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6712798b-0759-43a3-a5c6-a6c475ebc228-maas-api-tls\") pod \"6712798b-0759-43a3-a5c6-a6c475ebc228\" (UID: \"6712798b-0759-43a3-a5c6-a6c475ebc228\") "
Nov 02 12:14:38.239689 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.239552    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6712798b-0759-43a3-a5c6-a6c475ebc228-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6712798b-0759-43a3-a5c6-a6c475ebc228-openshift-service-ca\") pod \"6712798b-0759-43a3-a5c6-a6c475ebc228\" (UID: \"6712798b-0759-43a3-a5c6-a6c475ebc228\") "
Nov 02 12:14:38.240057 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.240022    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6712798b-0759-43a3-a5c6-a6c475ebc228-openshift-service-ca" pod "6712798b-0759-43a3-a5c6-a6c475ebc228" (UID: "6712798b-0759-43a3-a5c6-a6c475ebc228"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:14:38.241042 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.241017    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6712798b-0759-43a3-a5c6-a6c475ebc228-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:38.241994 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.241970    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6712798b-0759-43a3-a5c6-a6c475ebc228-maas-api-tls" pod "6712798b-0759-43a3-a5c6-a6c475ebc228" (UID: "6712798b-0759-43a3-a5c6-a6c475ebc228"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:14:38.242112 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.242082    4942 generic.go:356] "Generic (PLEG): container finished" podID="6712798b-0759-43a3-a5c6-a6c475ebc228" containerID="b12ceae0484ca2294cf2e91ef6f9da27cabde6889f78a2f317da79b385bbe263" exitCode=0
Nov 02 12:14:38.242156 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.242129    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw" event={"ID":"6712798b-0759-43a3-a5c6-a6c475ebc228","Type":"ContainerDied","Data":"b12ceae0484ca2294cf2e91ef6f9da27cabde6889f78a2f317da79b385bbe263"}
Nov 02 12:14:38.242156 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.242132    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6712798b-0759-43a3-a5c6-a6c475ebc228-kube-api-access-7h72x" pod "6712798b-0759-43a3-a5c6-a6c475ebc228" (UID: "6712798b-0759-43a3-a5c6-a6c475ebc228"). InnerVolumeSpecName "kube-api-access-7h72x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:14:38.242220 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.242205    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw" event={"ID":"6712798b-0759-43a3-a5c6-a6c475ebc228","Type":"ContainerDied","Data":"3c6c499dc009fa6a92d3dab486a9fd2e2f05df4cc67f067bfd3535b50ab023ad"}
Nov 02 12:14:38.242258 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.242228    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"
Nov 02 12:14:38.242258 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.242235    4942 scope.go:122] "RemoveContainer" containerID="b12ceae0484ca2294cf2e91ef6f9da27cabde6889f78a2f317da79b385bbe263"
Nov 02 12:14:38.242453 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.242425    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6712798b-0759-43a3-a5c6-a6c475ebc228-metrics-certs" pod "6712798b-0759-43a3-a5c6-a6c475ebc228" (UID: "6712798b-0759-43a3-a5c6-a6c475ebc228"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:14:38.290062 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.287703    4942 scope.go:122] "RemoveContainer" containerID="b12ceae0484ca2294cf2e91ef6f9da27cabde6889f78a2f317da79b385bbe263"
Nov 02 12:14:38.290351 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:14:38.290272    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b12ceae0484ca2294cf2e91ef6f9da27cabde6889f78a2f317da79b385bbe263\": container with ID starting with b12ceae0484ca2294cf2e91ef6f9da27cabde6889f78a2f317da79b385bbe263 not found: ID does not exist" containerID="b12ceae0484ca2294cf2e91ef6f9da27cabde6889f78a2f317da79b385bbe263"
Nov 02 12:14:38.290351 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.290329    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b12ceae0484ca2294cf2e91ef6f9da27cabde6889f78a2f317da79b385bbe263"} err="failed to get container status \"b12ceae0484ca2294cf2e91ef6f9da27cabde6889f78a2f317da79b385bbe263\": rpc error: code = NotFound desc = could not find container \"b12ceae0484ca2294cf2e91ef6f9da27cabde6889f78a2f317da79b385bbe263\": container with ID starting with b12ceae0484ca2294cf2e91ef6f9da27cabde6889f78a2f317da79b385bbe263 not found: ID does not exist"
Nov 02 12:14:38.342836 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.342805    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h72x\" (UniqueName: \"kubernetes.io/projected/6712798b-0759-43a3-a5c6-a6c475ebc228-kube-api-access-7h72x\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:38.343066 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.343038    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6712798b-0759-43a3-a5c6-a6c475ebc228-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:38.343192 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.343067    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6712798b-0759-43a3-a5c6-a6c475ebc228-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:38.665620 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.665587    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"]
Nov 02 12:14:38.670423 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:38.670391    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-966817-df4fd65cb-8jlgw"]
Nov 02 12:14:39.543479 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:39.543450    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6712798b-0759-43a3-a5c6-a6c475ebc228" path="/var/lib/kubelet/pods/6712798b-0759-43a3-a5c6-a6c475ebc228/volumes"
Nov 02 12:14:44.551040 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.551002    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"]
Nov 02 12:14:44.551544 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.551530    4942 cpu_manager.go:414] "containerMap: removing container" podUID="3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b" containerName="maas-api"
Nov 02 12:14:44.551591 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.551544    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b" containerName="maas-api"
Nov 02 12:14:44.551591 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.551580    4942 cpu_manager.go:414] "containerMap: removing container" podUID="6712798b-0759-43a3-a5c6-a6c475ebc228" containerName="maas-api"
Nov 02 12:14:44.551591 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.551585    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6712798b-0759-43a3-a5c6-a6c475ebc228" containerName="maas-api"
Nov 02 12:14:44.551691 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.551662    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ff3fe3d-cdc9-49ca-9ef6-89994a145a0b" containerName="maas-api"
Nov 02 12:14:44.551691 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.551675    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="6712798b-0759-43a3-a5c6-a6c475ebc228" containerName="maas-api"
Nov 02 12:14:44.556562 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.556540    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.646094 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.646058    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-workload-certs\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.646442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.646410    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istiod-ca-cert\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.646583 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.646560    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rpcj\" (UniqueName: \"kubernetes.io/projected/6e49a856-d29c-46d8-9606-cc2ba7bc9574-kube-api-access-4rpcj\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.646653 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.646602    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-credential-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.646714 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.646656    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-envoy\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.646786 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.646764    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-podinfo\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.646845 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.646803    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-token\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.646908 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.646849    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-data\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.646908 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.646887    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-workload-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.650853 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.650828    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"]
Nov 02 12:14:44.749006 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.748960    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-envoy\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.749215 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.749050    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-podinfo\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.749215 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.749080    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-token\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.749215 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.749119    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-data\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.749215 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.749149    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-workload-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.749215 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.749211    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-workload-certs\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.749572 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.749291    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istiod-ca-cert\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.749572 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.749368    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rpcj\" (UniqueName: \"kubernetes.io/projected/6e49a856-d29c-46d8-9606-cc2ba7bc9574-kube-api-access-4rpcj\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.749572 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.749393    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-credential-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.749853 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.749829    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-credential-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.763321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.762706    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-workload-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.786321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.770138    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-podinfo\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.786321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.770798    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-data\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.786321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.776676    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-workload-certs\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.786321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.783088    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istiod-ca-cert\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.787486 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.787444    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-envoy\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.830116 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.830048    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-token\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.838041 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.838015    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rpcj\" (UniqueName: \"kubernetes.io/projected/6e49a856-d29c-46d8-9606-cc2ba7bc9574-kube-api-access-4rpcj\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:44.870826 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:44.870797    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:45.304305 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:45.304268    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"]
Nov 02 12:14:45.306725 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:14:45.306698    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e49a856_d29c_46d8_9606_cc2ba7bc9574.slice/crio-7c08ba753dcde46afc3ff1ac2d56c1e44eb36ff67e6a43ee66e8704943f9530c WatchSource:0}: Error finding container 7c08ba753dcde46afc3ff1ac2d56c1e44eb36ff67e6a43ee66e8704943f9530c: Status 404 returned error can't find the container with id 7c08ba753dcde46afc3ff1ac2d56c1e44eb36ff67e6a43ee66e8704943f9530c
Nov 02 12:14:45.309063 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:45.309029    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:14:45.309136 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:45.309114    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:14:45.309178 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:45.309148    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:14:45.327899 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:45.327874    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w" event={"ID":"6e49a856-d29c-46d8-9606-cc2ba7bc9574","Type":"ContainerStarted","Data":"7c08ba753dcde46afc3ff1ac2d56c1e44eb36ff67e6a43ee66e8704943f9530c"}
Nov 02 12:14:46.336078 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:46.336043    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w" event={"ID":"6e49a856-d29c-46d8-9606-cc2ba7bc9574","Type":"ContainerStarted","Data":"2b0f621d86e7489e4a3bb413b2c058717d4e31b189b8460b75a067584fc4399d"}
Nov 02 12:14:46.471399 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:46.471319    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w" podStartSLOduration=2.471285198 podStartE2EDuration="2.471285198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:14:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:14:46.467882908 +0000 UTC m=+1113.039232663" watchObservedRunningTime="2026-11-02 12:14:46.471285198 +0000 UTC m=+1113.042634956"
Nov 02 12:14:46.873322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:46.872612    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:46.877437 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:46.877400    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 12:14:46.877599 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:46.877469    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w" podUID="6e49a856-d29c-46d8-9606-cc2ba7bc9574" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 12:14:47.120413 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:47.120380    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"]
Nov 02 12:14:47.120611 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:47.120582    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm" podUID="216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" containerName="istio-proxy" containerID="cri-o://28971369390a1457b619f972cbc11d58eaf2c0e10b864f0591a9fa08d0880822" gracePeriod=30
Nov 02 12:14:47.130818 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:47.130763    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"]
Nov 02 12:14:47.130954 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:47.130932    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6" podUID="72e2083e-9c67-4b7b-b583-7a1259ee4705" containerName="istio-proxy" containerID="cri-o://05ea3f519b19b7de9c67354a4affecdbbdfc77719b20bea828b0b9aac897a4c6" gracePeriod=30
Nov 02 12:14:47.185371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:47.185335    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"]
Nov 02 12:14:47.185614 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:47.185580    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd" podUID="73d2d79d-46c2-44c2-ae4a-3211e52244f4" containerName="istio-proxy" containerID="cri-o://e757a6685f93072f361a2b3e88cd6b0c17dfba245756f2a3714f5bfc7e0fef05" gracePeriod=30
Nov 02 12:14:47.191014 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:47.190985    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"]
Nov 02 12:14:47.191314 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:47.191263    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624" podUID="9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" containerName="istio-proxy" containerID="cri-o://fedc2ed2418fc64d4a44a13b5a204838132c812645d2309012647e3ddcfbcf8c" gracePeriod=30
Nov 02 12:14:47.877558 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:47.877520    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 12:14:47.878074 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:47.878027    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w" podUID="6e49a856-d29c-46d8-9606-cc2ba7bc9574" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 12:14:48.873321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:48.872559    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 12:14:48.873321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:48.872621    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w" podUID="6e49a856-d29c-46d8-9606-cc2ba7bc9574" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 12:14:49.876338 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:49.875336    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 12:14:49.876338 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:49.875409    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w" podUID="6e49a856-d29c-46d8-9606-cc2ba7bc9574" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 12:14:50.877323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:50.876682    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 12:14:50.877323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:50.876754    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w" podUID="6e49a856-d29c-46d8-9606-cc2ba7bc9574" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 12:14:51.871397 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:51.871359    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 02 12:14:51.871581 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:51.871428    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w" podUID="6e49a856-d29c-46d8-9606-cc2ba7bc9574" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 02 12:14:52.444501 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.444460    4942 generic.go:356] "Generic (PLEG): container finished" podID="73d2d79d-46c2-44c2-ae4a-3211e52244f4" containerID="e757a6685f93072f361a2b3e88cd6b0c17dfba245756f2a3714f5bfc7e0fef05" exitCode=0
Nov 02 12:14:52.444984 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.444585    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd" event={"ID":"73d2d79d-46c2-44c2-ae4a-3211e52244f4","Type":"ContainerDied","Data":"e757a6685f93072f361a2b3e88cd6b0c17dfba245756f2a3714f5bfc7e0fef05"}
Nov 02 12:14:52.447501 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.447341    4942 generic.go:356] "Generic (PLEG): container finished" podID="216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" containerID="28971369390a1457b619f972cbc11d58eaf2c0e10b864f0591a9fa08d0880822" exitCode=0
Nov 02 12:14:52.447635 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.447548    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm" event={"ID":"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1","Type":"ContainerDied","Data":"28971369390a1457b619f972cbc11d58eaf2c0e10b864f0591a9fa08d0880822"}
Nov 02 12:14:52.450993 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.449964    4942 generic.go:356] "Generic (PLEG): container finished" podID="9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" containerID="fedc2ed2418fc64d4a44a13b5a204838132c812645d2309012647e3ddcfbcf8c" exitCode=0
Nov 02 12:14:52.450993 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.450086    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624" event={"ID":"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42","Type":"ContainerDied","Data":"fedc2ed2418fc64d4a44a13b5a204838132c812645d2309012647e3ddcfbcf8c"}
Nov 02 12:14:52.450993 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.450134    4942 scope.go:122] "RemoveContainer" containerID="de2beb053e81c92e915186a39876f11c76d59c750d59ce755b3f3a9143a8ebf4"
Nov 02 12:14:52.461434 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.461355    4942 generic.go:356] "Generic (PLEG): container finished" podID="72e2083e-9c67-4b7b-b583-7a1259ee4705" containerID="05ea3f519b19b7de9c67354a4affecdbbdfc77719b20bea828b0b9aac897a4c6" exitCode=0
Nov 02 12:14:52.461894 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.461641    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6" event={"ID":"72e2083e-9c67-4b7b-b583-7a1259ee4705","Type":"ContainerDied","Data":"05ea3f519b19b7de9c67354a4affecdbbdfc77719b20bea828b0b9aac897a4c6"}
Nov 02 12:14:52.484962 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.484628    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:14:52.575371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.563266    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-kube-api-access-zxj47\" (UniqueName: \"kubernetes.io/projected/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-kube-api-access-zxj47\") pod \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") "
Nov 02 12:14:52.575371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.563377    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-workload-socket\") pod \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") "
Nov 02 12:14:52.575371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.563438    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-workload-certs\") pod \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") "
Nov 02 12:14:52.575371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.563480    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-token\" (UniqueName: \"kubernetes.io/projected/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-token\") pod \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") "
Nov 02 12:14:52.575371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.563534    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-credential-socket\") pod \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") "
Nov 02 12:14:52.575371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.563572    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-envoy\") pod \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") "
Nov 02 12:14:52.575371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.563623    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-data\") pod \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") "
Nov 02 12:14:52.575371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.563671    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istiod-ca-cert\") pod \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") "
Nov 02 12:14:52.575371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.563777    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-podinfo\") pod \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\" (UID: \"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1\") "
Nov 02 12:14:52.575371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.565563    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istiod-ca-cert" pod "216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" (UID: "216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:14:52.575371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.565815    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-workload-socket" pod "216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" (UID: "216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.575371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.565992    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-workload-certs" pod "216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" (UID: "216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.575371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.568946    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-credential-socket" pod "216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" (UID: "216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.575371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.575319    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-token" pod "216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" (UID: "216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:14:52.576239 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.575646    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-envoy" pod "216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" (UID: "216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.577758 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.577718    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-data" pod "216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" (UID: "216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.578963 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.578712    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-podinfo" pod "216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" (UID: "216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:14:52.578963 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.578748    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-kube-api-access-zxj47" pod "216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" (UID: "216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1"). InnerVolumeSpecName "kube-api-access-zxj47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:14:52.588950 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.588925    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:14:52.630665 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.630642    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:14:52.634623 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.634598    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:14:52.665080 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665047    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-envoy\") pod \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") "
Nov 02 12:14:52.665263 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665102    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-workload-socket\") pod \"72e2083e-9c67-4b7b-b583-7a1259ee4705\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") "
Nov 02 12:14:52.665263 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665132    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istiod-ca-cert\") pod \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") "
Nov 02 12:14:52.665263 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665177    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-credential-socket\") pod \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") "
Nov 02 12:14:52.665263 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665214    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-workload-certs\") pod \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") "
Nov 02 12:14:52.665263 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665259    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istiod-ca-cert\") pod \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") "
Nov 02 12:14:52.665579 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665285    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-envoy\") pod \"72e2083e-9c67-4b7b-b583-7a1259ee4705\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") "
Nov 02 12:14:52.665579 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665328    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-podinfo\") pod \"72e2083e-9c67-4b7b-b583-7a1259ee4705\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") "
Nov 02 12:14:52.665579 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665359    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-podinfo\") pod \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") "
Nov 02 12:14:52.665579 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665389    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/72e2083e-9c67-4b7b-b583-7a1259ee4705-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72e2083e-9c67-4b7b-b583-7a1259ee4705-istiod-ca-cert\") pod \"72e2083e-9c67-4b7b-b583-7a1259ee4705\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") "
Nov 02 12:14:52.665579 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665429    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-workload-socket\") pod \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") "
Nov 02 12:14:52.665579 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665463    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-workload-socket" pod "72e2083e-9c67-4b7b-b583-7a1259ee4705" (UID: "72e2083e-9c67-4b7b-b583-7a1259ee4705"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.665579 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665476    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-token\" (UniqueName: \"kubernetes.io/projected/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-token\") pod \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") "
Nov 02 12:14:52.665888 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665585    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-podinfo\") pod \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") "
Nov 02 12:14:52.665888 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665632    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-token\" (UniqueName: \"kubernetes.io/projected/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-token\") pod \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") "
Nov 02 12:14:52.665888 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665705    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-data\") pod \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") "
Nov 02 12:14:52.665888 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665758    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-credential-socket\") pod \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") "
Nov 02 12:14:52.665888 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665780    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-kube-api-access-gnzr6\" (UniqueName: \"kubernetes.io/projected/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-kube-api-access-gnzr6\") pod \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") "
Nov 02 12:14:52.665888 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665801    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/72e2083e-9c67-4b7b-b583-7a1259ee4705-kube-api-access-qgg67\" (UniqueName: \"kubernetes.io/projected/72e2083e-9c67-4b7b-b583-7a1259ee4705-kube-api-access-qgg67\") pod \"72e2083e-9c67-4b7b-b583-7a1259ee4705\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") "
Nov 02 12:14:52.665888 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665822    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73d2d79d-46c2-44c2-ae4a-3211e52244f4-kube-api-access-g8flh\" (UniqueName: \"kubernetes.io/projected/73d2d79d-46c2-44c2-ae4a-3211e52244f4-kube-api-access-g8flh\") pod \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") "
Nov 02 12:14:52.665888 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665841    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-credential-socket\") pod \"72e2083e-9c67-4b7b-b583-7a1259ee4705\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") "
Nov 02 12:14:52.666289 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665901    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-token\" (UniqueName: \"kubernetes.io/projected/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-token\") pod \"72e2083e-9c67-4b7b-b583-7a1259ee4705\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") "
Nov 02 12:14:52.666289 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665945    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-workload-certs\") pod \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\" (UID: \"73d2d79d-46c2-44c2-ae4a-3211e52244f4\") "
Nov 02 12:14:52.666289 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.665975    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-workload-certs\") pod \"72e2083e-9c67-4b7b-b583-7a1259ee4705\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") "
Nov 02 12:14:52.666289 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.666007    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-workload-socket\") pod \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") "
Nov 02 12:14:52.666289 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.666037    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-data\") pod \"72e2083e-9c67-4b7b-b583-7a1259ee4705\" (UID: \"72e2083e-9c67-4b7b-b583-7a1259ee4705\") "
Nov 02 12:14:52.666289 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.666084    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-data\") pod \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") "
Nov 02 12:14:52.666289 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.666125    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-envoy\") pod \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\" (UID: \"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42\") "
Nov 02 12:14:52.666735 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.666711    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.666804 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.666742    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.666804 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.666756    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.666804 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.666769    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.666804 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.666782    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.666804 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.666794    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.667046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.666807    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxj47\" (UniqueName: \"kubernetes.io/projected/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-kube-api-access-zxj47\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.667046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.666821    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.667046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.666835    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.667046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.666847    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.667814 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.667757    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72e2083e-9c67-4b7b-b583-7a1259ee4705-istiod-ca-cert" pod "72e2083e-9c67-4b7b-b583-7a1259ee4705" (UID: "72e2083e-9c67-4b7b-b583-7a1259ee4705"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:14:52.667814 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.667773    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-envoy" pod "73d2d79d-46c2-44c2-ae4a-3211e52244f4" (UID: "73d2d79d-46c2-44c2-ae4a-3211e52244f4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.668415 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.668334    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-token" pod "9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" (UID: "9c5fe5c5-f611-41ea-bc3f-f331df0c2c42"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:14:52.668647 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.668612    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-credential-socket" pod "9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" (UID: "9c5fe5c5-f611-41ea-bc3f-f331df0c2c42"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.668859 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.668836    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-workload-certs" pod "72e2083e-9c67-4b7b-b583-7a1259ee4705" (UID: "72e2083e-9c67-4b7b-b583-7a1259ee4705"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.669149 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.669127    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-credential-socket" pod "72e2083e-9c67-4b7b-b583-7a1259ee4705" (UID: "72e2083e-9c67-4b7b-b583-7a1259ee4705"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.670487 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.669544    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-podinfo" pod "9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" (UID: "9c5fe5c5-f611-41ea-bc3f-f331df0c2c42"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:14:52.670487 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.669553    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-envoy" pod "9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" (UID: "9c5fe5c5-f611-41ea-bc3f-f331df0c2c42"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.670487 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.669797    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-workload-certs" pod "9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" (UID: "9c5fe5c5-f611-41ea-bc3f-f331df0c2c42"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.670487 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.669830    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-workload-socket" pod "9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" (UID: "9c5fe5c5-f611-41ea-bc3f-f331df0c2c42"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.670487 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.670031    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-data" pod "72e2083e-9c67-4b7b-b583-7a1259ee4705" (UID: "72e2083e-9c67-4b7b-b583-7a1259ee4705"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.670487 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.670108    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istiod-ca-cert" pod "9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" (UID: "9c5fe5c5-f611-41ea-bc3f-f331df0c2c42"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:14:52.670487 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.670230    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-data" pod "9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" (UID: "9c5fe5c5-f611-41ea-bc3f-f331df0c2c42"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.671092 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.671069    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-kube-api-access-gnzr6" pod "9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" (UID: "9c5fe5c5-f611-41ea-bc3f-f331df0c2c42"). InnerVolumeSpecName "kube-api-access-gnzr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:14:52.671272 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.671241    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-envoy" pod "72e2083e-9c67-4b7b-b583-7a1259ee4705" (UID: "72e2083e-9c67-4b7b-b583-7a1259ee4705"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.671412 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.671386    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-token" pod "72e2083e-9c67-4b7b-b583-7a1259ee4705" (UID: "72e2083e-9c67-4b7b-b583-7a1259ee4705"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:14:52.672111 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.672085    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72e2083e-9c67-4b7b-b583-7a1259ee4705-kube-api-access-qgg67" pod "72e2083e-9c67-4b7b-b583-7a1259ee4705" (UID: "72e2083e-9c67-4b7b-b583-7a1259ee4705"). InnerVolumeSpecName "kube-api-access-qgg67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:14:52.672601 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.672545    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-credential-socket" pod "73d2d79d-46c2-44c2-ae4a-3211e52244f4" (UID: "73d2d79d-46c2-44c2-ae4a-3211e52244f4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.672741 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.672714    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-data" pod "73d2d79d-46c2-44c2-ae4a-3211e52244f4" (UID: "73d2d79d-46c2-44c2-ae4a-3211e52244f4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.672807 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.672733    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istiod-ca-cert" pod "73d2d79d-46c2-44c2-ae4a-3211e52244f4" (UID: "73d2d79d-46c2-44c2-ae4a-3211e52244f4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:14:52.672957 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.672936    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-workload-certs" pod "73d2d79d-46c2-44c2-ae4a-3211e52244f4" (UID: "73d2d79d-46c2-44c2-ae4a-3211e52244f4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.673706 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.673684    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-workload-socket" pod "73d2d79d-46c2-44c2-ae4a-3211e52244f4" (UID: "73d2d79d-46c2-44c2-ae4a-3211e52244f4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:14:52.674416 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.674388    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-podinfo" pod "72e2083e-9c67-4b7b-b583-7a1259ee4705" (UID: "72e2083e-9c67-4b7b-b583-7a1259ee4705"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:14:52.675071 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.675047    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-podinfo" pod "73d2d79d-46c2-44c2-ae4a-3211e52244f4" (UID: "73d2d79d-46c2-44c2-ae4a-3211e52244f4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:14:52.675281 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.675260    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-token" pod "73d2d79d-46c2-44c2-ae4a-3211e52244f4" (UID: "73d2d79d-46c2-44c2-ae4a-3211e52244f4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:14:52.676079 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.676052    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73d2d79d-46c2-44c2-ae4a-3211e52244f4-kube-api-access-g8flh" pod "73d2d79d-46c2-44c2-ae4a-3211e52244f4" (UID: "73d2d79d-46c2-44c2-ae4a-3211e52244f4"). InnerVolumeSpecName "kube-api-access-g8flh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:14:52.767687 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767599    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.767687 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767630    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.767687 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767643    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.767687 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767655    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.767687 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767664    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.767687 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767672    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.767687 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767682    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.767687 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767692    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.768253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767706    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.768253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767720    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.768253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767729    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.768253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767738    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.768253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767747    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.768253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767760    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72e2083e-9c67-4b7b-b583-7a1259ee4705-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.768253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767773    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.768253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767785    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72e2083e-9c67-4b7b-b583-7a1259ee4705-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.768253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767817    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.768253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767834    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.768253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767847    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.768253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767858    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.768253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767870    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73d2d79d-46c2-44c2-ae4a-3211e52244f4-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.768253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767905    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.768253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767920    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnzr6\" (UniqueName: \"kubernetes.io/projected/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42-kube-api-access-gnzr6\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.768253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767935    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgg67\" (UniqueName: \"kubernetes.io/projected/72e2083e-9c67-4b7b-b583-7a1259ee4705-kube-api-access-qgg67\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.768253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767952    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8flh\" (UniqueName: \"kubernetes.io/projected/73d2d79d-46c2-44c2-ae4a-3211e52244f4-kube-api-access-g8flh\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.768253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.767982    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e2083e-9c67-4b7b-b583-7a1259ee4705-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:14:52.875692 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.875659    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:52.875961 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.875937    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:52.876597 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:52.876577    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:14:53.470022 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:53.469937    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"
Nov 02 12:14:53.470022 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:53.469942    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6" event={"ID":"72e2083e-9c67-4b7b-b583-7a1259ee4705","Type":"ContainerDied","Data":"c268c576655862afb363acf4f03ae669f0ba6d972362a44eed725a700f034d9a"}
Nov 02 12:14:53.470022 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:53.470011    4942 scope.go:122] "RemoveContainer" containerID="05ea3f519b19b7de9c67354a4affecdbbdfc77719b20bea828b0b9aac897a4c6"
Nov 02 12:14:53.471546 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:53.471526    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"
Nov 02 12:14:53.471648 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:53.471546    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd" event={"ID":"73d2d79d-46c2-44c2-ae4a-3211e52244f4","Type":"ContainerDied","Data":"40f135d4e191419c50303f6c306ae728179958c1c1ac4140336b1450a525b93a"}
Nov 02 12:14:53.473404 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:53.473347    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm" event={"ID":"216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1","Type":"ContainerDied","Data":"07b65172e1c2e70ba617b4fd5e8a9207ea58ea9a10a60bac62047c6cbc5ee9ca"}
Nov 02 12:14:53.473525 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:53.473491    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"
Nov 02 12:14:53.475886 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:53.475858    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"
Nov 02 12:14:53.478425 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:53.478338    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624" event={"ID":"9c5fe5c5-f611-41ea-bc3f-f331df0c2c42","Type":"ContainerDied","Data":"d0a700b3b48f788bc6250f48ff166f5c41bdda9fe4c6094834df5ff60f8b70f2"}
Nov 02 12:14:53.486237 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:53.486204    4942 scope.go:122] "RemoveContainer" containerID="e757a6685f93072f361a2b3e88cd6b0c17dfba245756f2a3714f5bfc7e0fef05"
Nov 02 12:14:53.501816 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:53.501794    4942 scope.go:122] "RemoveContainer" containerID="28971369390a1457b619f972cbc11d58eaf2c0e10b864f0591a9fa08d0880822"
Nov 02 12:14:53.511175 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:53.511144    4942 scope.go:122] "RemoveContainer" containerID="fedc2ed2418fc64d4a44a13b5a204838132c812645d2309012647e3ddcfbcf8c"
Nov 02 12:14:53.635153 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:53.635125    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"]
Nov 02 12:14:53.686259 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:53.686225    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-zkvl6"]
Nov 02 12:14:53.793810 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:53.793720    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"]
Nov 02 12:14:53.799936 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:53.799908    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-sn8dd"]
Nov 02 12:14:53.908603 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:53.908537    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"]
Nov 02 12:14:53.913229 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:53.913203    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-mg624"]
Nov 02 12:14:54.022020 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:54.021993    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"]
Nov 02 12:14:54.025998 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:54.025962    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jhwsm"]
Nov 02 12:14:55.543921 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:55.543885    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" path="/var/lib/kubelet/pods/216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1/volumes"
Nov 02 12:14:55.544399 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:55.544367    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="72e2083e-9c67-4b7b-b583-7a1259ee4705" path="/var/lib/kubelet/pods/72e2083e-9c67-4b7b-b583-7a1259ee4705/volumes"
Nov 02 12:14:55.544836 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:55.544817    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73d2d79d-46c2-44c2-ae4a-3211e52244f4" path="/var/lib/kubelet/pods/73d2d79d-46c2-44c2-ae4a-3211e52244f4/volumes"
Nov 02 12:14:55.545962 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:55.545943    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" path="/var/lib/kubelet/pods/9c5fe5c5-f611-41ea-bc3f-f331df0c2c42/volumes"
Nov 02 12:14:57.453647 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.453616    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm"]
Nov 02 12:14:57.455014 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.454981    4942 cpu_manager.go:414] "containerMap: removing container" podUID="72e2083e-9c67-4b7b-b583-7a1259ee4705" containerName="istio-proxy"
Nov 02 12:14:57.455209 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.455191    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="72e2083e-9c67-4b7b-b583-7a1259ee4705" containerName="istio-proxy"
Nov 02 12:14:57.455402 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.455386    4942 cpu_manager.go:414] "containerMap: removing container" podUID="73d2d79d-46c2-44c2-ae4a-3211e52244f4" containerName="istio-proxy"
Nov 02 12:14:57.455507 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.455493    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73d2d79d-46c2-44c2-ae4a-3211e52244f4" containerName="istio-proxy"
Nov 02 12:14:57.455607 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.455595    4942 cpu_manager.go:414] "containerMap: removing container" podUID="9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" containerName="istio-proxy"
Nov 02 12:14:57.455694 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.455681    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" containerName="istio-proxy"
Nov 02 12:14:57.455793 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.455782    4942 cpu_manager.go:414] "containerMap: removing container" podUID="216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" containerName="istio-proxy"
Nov 02 12:14:57.455878 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.455866    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" containerName="istio-proxy"
Nov 02 12:14:57.456018 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.456006    4942 cpu_manager.go:414] "containerMap: removing container" podUID="9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" containerName="istio-proxy"
Nov 02 12:14:57.456108 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.456094    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" containerName="istio-proxy"
Nov 02 12:14:57.456337 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.456320    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="73d2d79d-46c2-44c2-ae4a-3211e52244f4" containerName="istio-proxy"
Nov 02 12:14:57.456457 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.456443    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" containerName="istio-proxy"
Nov 02 12:14:57.456547 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.456536    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="216e3c9b-1cf3-4b1e-9d49-a9ce3a3e5aa1" containerName="istio-proxy"
Nov 02 12:14:57.456648 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.456638    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c5fe5c5-f611-41ea-bc3f-f331df0c2c42" containerName="istio-proxy"
Nov 02 12:14:57.456733 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.456723    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="72e2083e-9c67-4b7b-b583-7a1259ee4705" containerName="istio-proxy"
Nov 02 12:14:57.803382 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.802428    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm"
Nov 02 12:14:57.875323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.870334    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-6ac03f\""
Nov 02 12:14:57.940795 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.912650    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6857980e-39b1-408b-9f98-71c8ec07677e-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm\" (UID: \"6857980e-39b1-408b-9f98-71c8ec07677e\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm"
Nov 02 12:14:57.940795 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.912732    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4x4m\" (UniqueName: \"kubernetes.io/projected/6857980e-39b1-408b-9f98-71c8ec07677e-kube-api-access-f4x4m\") pod \"payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm\" (UID: \"6857980e-39b1-408b-9f98-71c8ec07677e\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm"
Nov 02 12:14:57.998377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.996546    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm"]
Nov 02 12:14:57.998377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:57.998202    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5"]
Nov 02 12:14:58.012271 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:58.012242    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5"
Nov 02 12:14:58.027198 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:58.027143    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9jgh\" (UniqueName: \"kubernetes.io/projected/36599f9c-1da2-4ecf-a1ca-78382bc3ad9b-kube-api-access-h9jgh\") pod \"payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5\" (UID: \"36599f9c-1da2-4ecf-a1ca-78382bc3ad9b\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5"
Nov 02 12:14:58.027584 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:58.027562    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36599f9c-1da2-4ecf-a1ca-78382bc3ad9b-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5\" (UID: \"36599f9c-1da2-4ecf-a1ca-78382bc3ad9b\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5"
Nov 02 12:14:58.033346 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:58.033291    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6857980e-39b1-408b-9f98-71c8ec07677e-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm\" (UID: \"6857980e-39b1-408b-9f98-71c8ec07677e\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm"
Nov 02 12:14:58.045658 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:58.043168    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6857980e-39b1-408b-9f98-71c8ec07677e-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm\" (UID: \"6857980e-39b1-408b-9f98-71c8ec07677e\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm"
Nov 02 12:14:58.045942 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:58.045910    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4x4m\" (UniqueName: \"kubernetes.io/projected/6857980e-39b1-408b-9f98-71c8ec07677e-kube-api-access-f4x4m\") pod \"payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm\" (UID: \"6857980e-39b1-408b-9f98-71c8ec07677e\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm"
Nov 02 12:14:58.069368 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:58.058714    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5"]
Nov 02 12:14:58.082776 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:58.082746    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-6ac03f-dockercfg-92r2h\""
Nov 02 12:14:58.139322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:58.136988    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4x4m\" (UniqueName: \"kubernetes.io/projected/6857980e-39b1-408b-9f98-71c8ec07677e-kube-api-access-f4x4m\") pod \"payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm\" (UID: \"6857980e-39b1-408b-9f98-71c8ec07677e\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm"
Nov 02 12:14:58.146328 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:58.145387    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8" podUID="e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 02 12:14:58.148357 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:58.148330    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9jgh\" (UniqueName: \"kubernetes.io/projected/36599f9c-1da2-4ecf-a1ca-78382bc3ad9b-kube-api-access-h9jgh\") pod \"payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5\" (UID: \"36599f9c-1da2-4ecf-a1ca-78382bc3ad9b\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5"
Nov 02 12:14:58.177220 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:58.177187    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36599f9c-1da2-4ecf-a1ca-78382bc3ad9b-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5\" (UID: \"36599f9c-1da2-4ecf-a1ca-78382bc3ad9b\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5"
Nov 02 12:14:58.178741 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:58.178717    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36599f9c-1da2-4ecf-a1ca-78382bc3ad9b-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5\" (UID: \"36599f9c-1da2-4ecf-a1ca-78382bc3ad9b\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5"
Nov 02 12:14:58.269322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:58.251092    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9jgh\" (UniqueName: \"kubernetes.io/projected/36599f9c-1da2-4ecf-a1ca-78382bc3ad9b-kube-api-access-h9jgh\") pod \"payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5\" (UID: \"36599f9c-1da2-4ecf-a1ca-78382bc3ad9b\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5"
Nov 02 12:14:58.296881 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:58.295616    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm"
Nov 02 12:14:58.410333 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:58.407516    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5"
Nov 02 12:14:59.151476 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:59.151428    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm"]
Nov 02 12:14:59.394693 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:59.394662    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5"]
Nov 02 12:14:59.395798 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:14:59.395770    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36599f9c_1da2_4ecf_a1ca_78382bc3ad9b.slice/crio-b3e199e4f2c03ace773ded9a815d765c131d308487df5a4a2daa11aacc651ef2 WatchSource:0}: Error finding container b3e199e4f2c03ace773ded9a815d765c131d308487df5a4a2daa11aacc651ef2: Status 404 returned error can't find the container with id b3e199e4f2c03ace773ded9a815d765c131d308487df5a4a2daa11aacc651ef2
Nov 02 12:14:59.792672 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:59.792573    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm" event={"ID":"6857980e-39b1-408b-9f98-71c8ec07677e","Type":"ContainerStarted","Data":"ad41dfe2d0231cde8602ab736e52a78af41d9600632777df7c3c9eea61b2ffde"}
Nov 02 12:14:59.792672 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:59.792645    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm" event={"ID":"6857980e-39b1-408b-9f98-71c8ec07677e","Type":"ContainerStarted","Data":"1dd719c89777dd06bc41daef31707952c0feacfaea98d936b72250eb669e33ed"}
Nov 02 12:14:59.792891 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:59.792796    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm"
Nov 02 12:14:59.794895 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:59.794870    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5" event={"ID":"36599f9c-1da2-4ecf-a1ca-78382bc3ad9b","Type":"ContainerStarted","Data":"d4fd307346542846e73bd33905803e891e1cfa52cb4837dcfca377a9966d6f52"}
Nov 02 12:14:59.795049 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:59.794910    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5" event={"ID":"36599f9c-1da2-4ecf-a1ca-78382bc3ad9b","Type":"ContainerStarted","Data":"b3e199e4f2c03ace773ded9a815d765c131d308487df5a4a2daa11aacc651ef2"}
Nov 02 12:14:59.795049 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:59.794930    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5"
Nov 02 12:14:59.909692 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:14:59.909616    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm" podStartSLOduration=2.909595815 podStartE2EDuration="2.909595815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:14:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:14:59.907854459 +0000 UTC m=+1126.479204218" watchObservedRunningTime="2026-11-02 12:14:59.909595815 +0000 UTC m=+1126.480945573"
Nov 02 12:15:00.041050 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:00.040974    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5" podStartSLOduration=3.04095073 podStartE2EDuration="3.04095073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:14:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:15:00.029501376 +0000 UTC m=+1126.600851136" watchObservedRunningTime="2026-11-02 12:15:00.04095073 +0000 UTC m=+1126.612300488"
Nov 02 12:15:00.208133 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:00.208098    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c"]
Nov 02 12:15:00.212457 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:00.212434    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c"
Nov 02 12:15:00.296318 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:00.294275    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c"]
Nov 02 12:15:00.346337 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:00.343999    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nbw2z\""
Nov 02 12:15:00.348703 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:00.348673    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9959\" (UniqueName: \"kubernetes.io/projected/f0d799e3-5a03-4723-84cc-c0e6e819ba9b-kube-api-access-b9959\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c\" (UID: \"f0d799e3-5a03-4723-84cc-c0e6e819ba9b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c"
Nov 02 12:15:00.348872 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:00.348752    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0d799e3-5a03-4723-84cc-c0e6e819ba9b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c\" (UID: \"f0d799e3-5a03-4723-84cc-c0e6e819ba9b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c"
Nov 02 12:15:00.449713 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:00.449675    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0d799e3-5a03-4723-84cc-c0e6e819ba9b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c\" (UID: \"f0d799e3-5a03-4723-84cc-c0e6e819ba9b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c"
Nov 02 12:15:00.449895 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:00.449844    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9959\" (UniqueName: \"kubernetes.io/projected/f0d799e3-5a03-4723-84cc-c0e6e819ba9b-kube-api-access-b9959\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c\" (UID: \"f0d799e3-5a03-4723-84cc-c0e6e819ba9b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c"
Nov 02 12:15:00.450363 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:00.450342    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0d799e3-5a03-4723-84cc-c0e6e819ba9b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c\" (UID: \"f0d799e3-5a03-4723-84cc-c0e6e819ba9b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c"
Nov 02 12:15:00.510682 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:00.510610    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9959\" (UniqueName: \"kubernetes.io/projected/f0d799e3-5a03-4723-84cc-c0e6e819ba9b-kube-api-access-b9959\") pod \"maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c\" (UID: \"f0d799e3-5a03-4723-84cc-c0e6e819ba9b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c"
Nov 02 12:15:00.526434 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:00.526408    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c"
Nov 02 12:15:00.910463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:00.910431    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c"]
Nov 02 12:15:00.911012 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:15:00.910982    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0d799e3_5a03_4723_84cc_c0e6e819ba9b.slice/crio-b3b915f75d0df591ab3d1435f6c1e49ca478ed7b848f874ecc945b79ef9631e4 WatchSource:0}: Error finding container b3b915f75d0df591ab3d1435f6c1e49ca478ed7b848f874ecc945b79ef9631e4: Status 404 returned error can't find the container with id b3b915f75d0df591ab3d1435f6c1e49ca478ed7b848f874ecc945b79ef9631e4
Nov 02 12:15:00.985631 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:15:00.985587    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:15:01.808686 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:01.808654    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c" event={"ID":"f0d799e3-5a03-4723-84cc-c0e6e819ba9b","Type":"ContainerStarted","Data":"b3b915f75d0df591ab3d1435f6c1e49ca478ed7b848f874ecc945b79ef9631e4"}
Nov 02 12:15:04.829759 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:04.829714    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c" event={"ID":"f0d799e3-5a03-4723-84cc-c0e6e819ba9b","Type":"ContainerStarted","Data":"719a70f8c4078378f10d5256f99b2e41ae2adc54ec5757ed9024f5b7314735ae"}
Nov 02 12:15:04.945728 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:04.945630    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c" podStartSLOduration=1.500935895 podStartE2EDuration="4.945615069s" totalImagesPullingTime="3.444679174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:15:04.944186675 +0000 UTC m=+1131.515536433" watchObservedRunningTime="2026-11-02 12:15:04.945615069 +0000 UTC m=+1131.516964853"
Nov 02 12:15:05.837701 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:05.837615    4942 generic.go:356] "Generic (PLEG): container finished" podID="f0d799e3-5a03-4723-84cc-c0e6e819ba9b" containerID="719a70f8c4078378f10d5256f99b2e41ae2adc54ec5757ed9024f5b7314735ae" exitCode=60
Nov 02 12:15:05.837701 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:05.837651    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c" event={"ID":"f0d799e3-5a03-4723-84cc-c0e6e819ba9b","Type":"ContainerDied","Data":"719a70f8c4078378f10d5256f99b2e41ae2adc54ec5757ed9024f5b7314735ae"}
Nov 02 12:15:05.838091 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:05.837894    4942 scope.go:122] "RemoveContainer" containerID="719a70f8c4078378f10d5256f99b2e41ae2adc54ec5757ed9024f5b7314735ae"
Nov 02 12:15:06.845829 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:06.845794    4942 generic.go:356] "Generic (PLEG): container finished" podID="f0d799e3-5a03-4723-84cc-c0e6e819ba9b" containerID="ed2eb45511c699b7a92ed1beddc5a054ab1613d51d9ee1b9588c2e75764a13c0" exitCode=60
Nov 02 12:15:06.846247 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:06.845861    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c" event={"ID":"f0d799e3-5a03-4723-84cc-c0e6e819ba9b","Type":"ContainerDied","Data":"ed2eb45511c699b7a92ed1beddc5a054ab1613d51d9ee1b9588c2e75764a13c0"}
Nov 02 12:15:06.846247 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:06.845934    4942 scope.go:122] "RemoveContainer" containerID="719a70f8c4078378f10d5256f99b2e41ae2adc54ec5757ed9024f5b7314735ae"
Nov 02 12:15:06.846247 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:06.846163    4942 scope.go:122] "RemoveContainer" containerID="ed2eb45511c699b7a92ed1beddc5a054ab1613d51d9ee1b9588c2e75764a13c0"
Nov 02 12:15:06.846502 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:15:06.846478    4942 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c_odh-ai-gateway-infra(f0d799e3-5a03-4723-84cc-c0e6e819ba9b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c" podUID="f0d799e3-5a03-4723-84cc-c0e6e819ba9b"
Nov 02 12:15:07.833178 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:07.833146    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw"]
Nov 02 12:15:07.838172 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:07.838155    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw"
Nov 02 12:15:07.842952 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:07.842925    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdwtc\" (UniqueName: \"kubernetes.io/projected/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-kube-api-access-tdwtc\") pod \"maas-api-revoke-sub-68a5a075-nvfzw\" (UID: \"62c1a154-3a84-4f8b-bbc1-f4c28740ee31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw"
Nov 02 12:15:07.843073 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:07.842975    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-68a5a075-nvfzw\" (UID: \"62c1a154-3a84-4f8b-bbc1-f4c28740ee31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw"
Nov 02 12:15:07.843122 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:07.843102    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-maas-api-xks-ca\") pod \"maas-api-revoke-sub-68a5a075-nvfzw\" (UID: \"62c1a154-3a84-4f8b-bbc1-f4c28740ee31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw"
Nov 02 12:15:07.855854 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:07.855834    4942 scope.go:122] "RemoveContainer" containerID="ed2eb45511c699b7a92ed1beddc5a054ab1613d51d9ee1b9588c2e75764a13c0"
Nov 02 12:15:07.856139 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:15:07.856090    4942 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c_odh-ai-gateway-infra(f0d799e3-5a03-4723-84cc-c0e6e819ba9b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c" podUID="f0d799e3-5a03-4723-84cc-c0e6e819ba9b"
Nov 02 12:15:07.894353 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:07.894204    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw"]
Nov 02 12:15:07.894353 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:07.894352    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 12:15:07.943732 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:07.943704    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-68a5a075-nvfzw\" (UID: \"62c1a154-3a84-4f8b-bbc1-f4c28740ee31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw"
Nov 02 12:15:07.943868 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:07.943742    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-maas-api-xks-ca\") pod \"maas-api-revoke-sub-68a5a075-nvfzw\" (UID: \"62c1a154-3a84-4f8b-bbc1-f4c28740ee31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw"
Nov 02 12:15:07.943918 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:07.943890    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdwtc\" (UniqueName: \"kubernetes.io/projected/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-kube-api-access-tdwtc\") pod \"maas-api-revoke-sub-68a5a075-nvfzw\" (UID: \"62c1a154-3a84-4f8b-bbc1-f4c28740ee31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw"
Nov 02 12:15:07.944525 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:07.944493    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-68a5a075-nvfzw\" (UID: \"62c1a154-3a84-4f8b-bbc1-f4c28740ee31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw"
Nov 02 12:15:07.946089 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:07.946068    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-maas-api-xks-ca\") pod \"maas-api-revoke-sub-68a5a075-nvfzw\" (UID: \"62c1a154-3a84-4f8b-bbc1-f4c28740ee31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw"
Nov 02 12:15:08.009290 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:08.009256    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdwtc\" (UniqueName: \"kubernetes.io/projected/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-kube-api-access-tdwtc\") pod \"maas-api-revoke-sub-68a5a075-nvfzw\" (UID: \"62c1a154-3a84-4f8b-bbc1-f4c28740ee31\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw"
Nov 02 12:15:08.155331 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:08.155269    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw"
Nov 02 12:15:08.536227 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:08.536196    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw"]
Nov 02 12:15:08.537847 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:15:08.537814    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62c1a154_3a84_4f8b_bbc1_f4c28740ee31.slice/crio-6dc42bb8a2aca497549e7f6c374c907f1f39b0fc8cf3d394f8d02e813041d3c5 WatchSource:0}: Error finding container 6dc42bb8a2aca497549e7f6c374c907f1f39b0fc8cf3d394f8d02e813041d3c5: Status 404 returned error can't find the container with id 6dc42bb8a2aca497549e7f6c374c907f1f39b0fc8cf3d394f8d02e813041d3c5
Nov 02 12:15:08.862813 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:08.862784    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw" event={"ID":"62c1a154-3a84-4f8b-bbc1-f4c28740ee31","Type":"ContainerStarted","Data":"6dc42bb8a2aca497549e7f6c374c907f1f39b0fc8cf3d394f8d02e813041d3c5"}
Nov 02 12:15:09.870239 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:09.870202    4942 generic.go:356] "Generic (PLEG): container finished" podID="62c1a154-3a84-4f8b-bbc1-f4c28740ee31" containerID="3a271b864097048db44f80afb55300c3dec54e6c8880585b57ad8d24ef3610ee" exitCode=0
Nov 02 12:15:09.870712 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:09.870287    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw" event={"ID":"62c1a154-3a84-4f8b-bbc1-f4c28740ee31","Type":"ContainerDied","Data":"3a271b864097048db44f80afb55300c3dec54e6c8880585b57ad8d24ef3610ee"}
Nov 02 12:15:10.805040 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:10.805006    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm"
Nov 02 12:15:10.805210 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:10.805081    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5"
Nov 02 12:15:11.030487 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.030461    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw"
Nov 02 12:15:11.084049 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.083966    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-maas-api-xks-ca\") pod \"62c1a154-3a84-4f8b-bbc1-f4c28740ee31\" (UID: \"62c1a154-3a84-4f8b-bbc1-f4c28740ee31\") "
Nov 02 12:15:11.084196 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.084106    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-maas-api-openshift-ca\") pod \"62c1a154-3a84-4f8b-bbc1-f4c28740ee31\" (UID: \"62c1a154-3a84-4f8b-bbc1-f4c28740ee31\") "
Nov 02 12:15:11.084250 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.084234    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-kube-api-access-tdwtc\" (UniqueName: \"kubernetes.io/projected/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-kube-api-access-tdwtc\") pod \"62c1a154-3a84-4f8b-bbc1-f4c28740ee31\" (UID: \"62c1a154-3a84-4f8b-bbc1-f4c28740ee31\") "
Nov 02 12:15:11.084570 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.084519    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-maas-api-openshift-ca" pod "62c1a154-3a84-4f8b-bbc1-f4c28740ee31" (UID: "62c1a154-3a84-4f8b-bbc1-f4c28740ee31"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:15:11.086357 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.086284    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-kube-api-access-tdwtc" pod "62c1a154-3a84-4f8b-bbc1-f4c28740ee31" (UID: "62c1a154-3a84-4f8b-bbc1-f4c28740ee31"). InnerVolumeSpecName "kube-api-access-tdwtc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:15:11.086574 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.086549    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-maas-api-xks-ca" pod "62c1a154-3a84-4f8b-bbc1-f4c28740ee31" (UID: "62c1a154-3a84-4f8b-bbc1-f4c28740ee31"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:15:11.185607 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.185576    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-maas-api-openshift-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:15:11.185607 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.185601    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdwtc\" (UniqueName: \"kubernetes.io/projected/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-kube-api-access-tdwtc\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:15:11.185607 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.185611    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/62c1a154-3a84-4f8b-bbc1-f4c28740ee31-maas-api-xks-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:15:11.882763 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.882728    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"]
Nov 02 12:15:11.883432 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.883410    4942 cpu_manager.go:414] "containerMap: removing container" podUID="62c1a154-3a84-4f8b-bbc1-f4c28740ee31" containerName="revoke-keys"
Nov 02 12:15:11.883432 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.883431    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="62c1a154-3a84-4f8b-bbc1-f4c28740ee31" containerName="revoke-keys"
Nov 02 12:15:11.883625 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.883616    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="62c1a154-3a84-4f8b-bbc1-f4c28740ee31" containerName="revoke-keys"
Nov 02 12:15:11.884470 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.884448    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw"
Nov 02 12:15:11.888533 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.888509    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw" event={"ID":"62c1a154-3a84-4f8b-bbc1-f4c28740ee31","Type":"ContainerDied","Data":"6dc42bb8a2aca497549e7f6c374c907f1f39b0fc8cf3d394f8d02e813041d3c5"}
Nov 02 12:15:11.888653 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.888536    4942 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6dc42bb8a2aca497549e7f6c374c907f1f39b0fc8cf3d394f8d02e813041d3c5"
Nov 02 12:15:11.888653 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.888612    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:11.943402 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.943372    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-036f88-openshift-default-dockercfg-gfh5b\""
Nov 02 12:15:11.993827 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.993796    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-workload-socket\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:11.994028 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.994010    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istiod-ca-cert\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:11.994196 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.994170    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-envoy\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:11.994362 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.994068    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"]
Nov 02 12:15:11.994362 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.994260    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-token\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:11.994446 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.994389    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw42d\" (UniqueName: \"kubernetes.io/projected/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-kube-api-access-nw42d\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:11.994501 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.994477    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-data\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:11.994558 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.994508    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-podinfo\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:11.994558 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.994539    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-workload-certs\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:11.994630 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:11.994578    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-credential-socket\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.095544 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.095512    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-workload-socket\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.095544 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.095547    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istiod-ca-cert\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.096159 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.095643    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-envoy\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.096159 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.095677    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-token\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.096159 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.095726    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw42d\" (UniqueName: \"kubernetes.io/projected/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-kube-api-access-nw42d\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.096159 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.095793    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-data\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.096159 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.095820    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-podinfo\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.096159 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.095852    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-workload-certs\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.096159 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.095894    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-credential-socket\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.096159 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.096000    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-workload-socket\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.096600 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.096446    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istiod-ca-cert\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.096600 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.096550    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-credential-socket\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.096740 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.096716    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-data\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.096798 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.096721    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-workload-certs\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.098455 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.098425    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-envoy\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.098455 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.098448    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-podinfo\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.157052 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.156990    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-token\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.157052 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.157028    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw42d\" (UniqueName: \"kubernetes.io/projected/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-kube-api-access-nw42d\") pod \"e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") " pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.201763 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.201734    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:12.632679 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.632652    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"]
Nov 02 12:15:12.633808 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:15:12.633783    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37f6c8a7_d9a5_4d1d_b46a_8ec120b1cea3.slice/crio-b7271e450c31cff0a5a6fa24bdd25ed86b2f779eeb1f82adf667b7b48e4575cf WatchSource:0}: Error finding container b7271e450c31cff0a5a6fa24bdd25ed86b2f779eeb1f82adf667b7b48e4575cf: Status 404 returned error can't find the container with id b7271e450c31cff0a5a6fa24bdd25ed86b2f779eeb1f82adf667b7b48e4575cf
Nov 02 12:15:12.636270 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.636236    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:15:12.636374 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.636319    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:15:12.636374 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.636349    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:15:12.897365 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.896425    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62" event={"ID":"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3","Type":"ContainerStarted","Data":"e48ba6c6d932d48fa85f15a7b0364f69a2296dffd5e2e7171c146bcdf62cf321"}
Nov 02 12:15:12.897365 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:12.896475    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62" event={"ID":"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3","Type":"ContainerStarted","Data":"b7271e450c31cff0a5a6fa24bdd25ed86b2f779eeb1f82adf667b7b48e4575cf"}
Nov 02 12:15:13.017238 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:13.017190    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62" podStartSLOduration=2.017173885 podStartE2EDuration="2.017173885s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:15:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:15:13.016728439 +0000 UTC m=+1139.588078208" watchObservedRunningTime="2026-11-02 12:15:13.017173885 +0000 UTC m=+1139.588523643"
Nov 02 12:15:13.202051 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:13.201977    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:13.203901 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:13.203870    4942 patch_prober.go:29] interesting pod/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 12:15:13.204028 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:13.203920    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62" podUID="37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 12:15:14.202969 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:14.202925    4942 patch_prober.go:29] interesting pod/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 12:15:14.203389 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:14.202980    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62" podUID="37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 12:15:14.809323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:14.807482    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"]
Nov 02 12:15:14.835326 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:14.825967    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:14.911324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:14.908677    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"]
Nov 02 12:15:15.079381 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.054609    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-credential-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.079677 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.079646    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm684\" (UniqueName: \"kubernetes.io/projected/5596075f-9595-476d-a6c2-45234fa26395-kube-api-access-lm684\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.079875 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.079856    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5596075f-9595-476d-a6c2-45234fa26395-istiod-ca-cert\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.080062 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.080045    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5596075f-9595-476d-a6c2-45234fa26395-istio-podinfo\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.080166 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.080152    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-workload-certs\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.080338 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.080322    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-istio-data\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.098432 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.098394    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-istio-envoy\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.098662 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.098641    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5596075f-9595-476d-a6c2-45234fa26395-istio-token\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.102316 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.098825    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-workload-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.208469    4942 patch_prober.go:29] interesting pod/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 12:15:15.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.208530    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62" podUID="37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 12:15:15.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.209449    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5596075f-9595-476d-a6c2-45234fa26395-istiod-ca-cert\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.209491    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5596075f-9595-476d-a6c2-45234fa26395-istio-podinfo\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.209520    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-workload-certs\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.209575    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-istio-data\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.209621    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-istio-envoy\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.209655    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5596075f-9595-476d-a6c2-45234fa26395-istio-token\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.209715    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-workload-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.209798    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-credential-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.209826    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm684\" (UniqueName: \"kubernetes.io/projected/5596075f-9595-476d-a6c2-45234fa26395-kube-api-access-lm684\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.233705    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-workload-certs\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.233983    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-istio-data\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.248729    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5596075f-9595-476d-a6c2-45234fa26395-istiod-ca-cert\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.249727    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-workload-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.250313    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-credential-socket\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.268638    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-istio-envoy\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.297125    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5596075f-9595-476d-a6c2-45234fa26395-istio-podinfo\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.371124 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.371084    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5596075f-9595-476d-a6c2-45234fa26395-istio-token\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.452213 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.452141    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm684\" (UniqueName: \"kubernetes.io/projected/5596075f-9595-476d-a6c2-45234fa26395-kube-api-access-lm684\") pod \"e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") " pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:15.491517 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:15.491483    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:16.157220 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:16.156260    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8" podUID="e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:15:16.265515 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:16.227428    4942 patch_prober.go:29] interesting pod/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 12:15:16.265515 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:16.227489    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62" podUID="37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 12:15:16.875324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:16.858524    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f" podUID="5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 12:15:17.019364 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:17.019318    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw" podUID="242aa9d0-49d5-4981-ad74-0f7b95635e6a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:15:17.097534 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:17.096909    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:15:17.100920 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:17.100854    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:15:17.101235 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:17.101204    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:15:17.203432 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:17.201009    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8" podUID="e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:15:17.238331 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:17.230543    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:15:17.254334 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:17.240271    4942 patch_prober.go:29] interesting pod/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 12:15:17.254334 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:17.240353    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62" podUID="37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 12:15:17.361347 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:17.337353    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"]
Nov 02 12:15:17.620385 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:17.544406    4942 scope.go:122] "RemoveContainer" containerID="ed2eb45511c699b7a92ed1beddc5a054ab1613d51d9ee1b9588c2e75764a13c0"
Nov 02 12:15:17.620385 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:17.605625    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk" event={"ID":"5596075f-9595-476d-a6c2-45234fa26395","Type":"ContainerStarted","Data":"0d1cd65dcf60e58b52f444da2d3c5bd08fcb65efc06ff0a9d4247779f5dc56a1"}
Nov 02 12:15:17.874794 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:17.874700    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f" podUID="5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:15:17.875759 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:17.875733    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:15:18.042622 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:18.042578    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw" podUID="242aa9d0-49d5-4981-ad74-0f7b95635e6a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:15:18.043886 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:18.043861    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:15:18.166255 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:18.166227    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:15:18.207843 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:18.207808    4942 patch_prober.go:29] interesting pod/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 12:15:18.208115 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:18.208085    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62" podUID="37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 12:15:18.208650 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:18.208618    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8" podUID="e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:15:18.235092 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:18.235062    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:15:18.322378 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:18.322284    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:15:18.614072 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:18.614036    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c" event={"ID":"f0d799e3-5a03-4723-84cc-c0e6e819ba9b","Type":"ContainerStarted","Data":"38b5bd45aa96ca9a073a82167d57ba166fd4bfaf0a7bd64205a0c496b533639d"}
Nov 02 12:15:18.615985 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:18.615948    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk" event={"ID":"5596075f-9595-476d-a6c2-45234fa26395","Type":"ContainerStarted","Data":"b07fb56af6453ba38db180a95d7016426ec0afb8df951bc4bc898b481757d18c"}
Nov 02 12:15:18.919320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:18.918872    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk" podStartSLOduration=4.918856754 podStartE2EDuration="4.918856754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:15:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:15:18.798668486 +0000 UTC m=+1145.370018244" watchObservedRunningTime="2026-11-02 12:15:18.918856754 +0000 UTC m=+1145.490206510"
Nov 02 12:15:19.211366 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:19.207711    4942 patch_prober.go:29] interesting pod/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 12:15:19.211366 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:19.207777    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62" podUID="37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 12:15:19.496640 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:19.494522    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:19.496640 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:19.496237    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 12:15:19.496640 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:19.496313    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk" podUID="5596075f-9595-476d-a6c2-45234fa26395" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 12:15:19.627278 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:19.627241    4942 generic.go:356] "Generic (PLEG): container finished" podID="f0d799e3-5a03-4723-84cc-c0e6e819ba9b" containerID="38b5bd45aa96ca9a073a82167d57ba166fd4bfaf0a7bd64205a0c496b533639d" exitCode=60
Nov 02 12:15:19.627488 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:19.627383    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c" event={"ID":"f0d799e3-5a03-4723-84cc-c0e6e819ba9b","Type":"ContainerDied","Data":"38b5bd45aa96ca9a073a82167d57ba166fd4bfaf0a7bd64205a0c496b533639d"}
Nov 02 12:15:19.627566 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:19.627491    4942 scope.go:122] "RemoveContainer" containerID="ed2eb45511c699b7a92ed1beddc5a054ab1613d51d9ee1b9588c2e75764a13c0"
Nov 02 12:15:19.627959 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:19.627935    4942 scope.go:122] "RemoveContainer" containerID="38b5bd45aa96ca9a073a82167d57ba166fd4bfaf0a7bd64205a0c496b533639d"
Nov 02 12:15:19.628221 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:15:19.628197    4942 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c_odh-ai-gateway-infra(f0d799e3-5a03-4723-84cc-c0e6e819ba9b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c" podUID="f0d799e3-5a03-4723-84cc-c0e6e819ba9b"
Nov 02 12:15:19.927434 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:19.927401    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c"]
Nov 02 12:15:20.202282 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:20.202176    4942 patch_prober.go:29] interesting pod/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 02 12:15:20.202282 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:20.202253    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62" podUID="37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 02 12:15:20.493259 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:20.493174    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 12:15:20.493484 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:20.493461    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk" podUID="5596075f-9595-476d-a6c2-45234fa26395" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 12:15:20.842999 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:20.842972    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c"
Nov 02 12:15:20.950202 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:20.950170    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0d799e3-5a03-4723-84cc-c0e6e819ba9b-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0d799e3-5a03-4723-84cc-c0e6e819ba9b-maas-api-service-ca\") pod \"f0d799e3-5a03-4723-84cc-c0e6e819ba9b\" (UID: \"f0d799e3-5a03-4723-84cc-c0e6e819ba9b\") "
Nov 02 12:15:20.950664 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:20.950230    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0d799e3-5a03-4723-84cc-c0e6e819ba9b-kube-api-access-b9959\" (UniqueName: \"kubernetes.io/projected/f0d799e3-5a03-4723-84cc-c0e6e819ba9b-kube-api-access-b9959\") pod \"f0d799e3-5a03-4723-84cc-c0e6e819ba9b\" (UID: \"f0d799e3-5a03-4723-84cc-c0e6e819ba9b\") "
Nov 02 12:15:20.950664 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:20.950603    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0d799e3-5a03-4723-84cc-c0e6e819ba9b-maas-api-service-ca" pod "f0d799e3-5a03-4723-84cc-c0e6e819ba9b" (UID: "f0d799e3-5a03-4723-84cc-c0e6e819ba9b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:15:20.950780 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:20.950764    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0d799e3-5a03-4723-84cc-c0e6e819ba9b-maas-api-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:15:20.958338 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:20.952282    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0d799e3-5a03-4723-84cc-c0e6e819ba9b-kube-api-access-b9959" pod "f0d799e3-5a03-4723-84cc-c0e6e819ba9b" (UID: "f0d799e3-5a03-4723-84cc-c0e6e819ba9b"). InnerVolumeSpecName "kube-api-access-b9959". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:15:21.052290 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:21.052258    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9959\" (UniqueName: \"kubernetes.io/projected/f0d799e3-5a03-4723-84cc-c0e6e819ba9b-kube-api-access-b9959\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:15:21.206647 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:21.206618    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:21.206933 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:21.206908    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:21.207629 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:21.207608    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:15:21.492940 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:21.492845    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 12:15:21.493209 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:21.493172    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk" podUID="5596075f-9595-476d-a6c2-45234fa26395" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 12:15:21.647322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:21.647283    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c"
Nov 02 12:15:21.647486 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:21.647331    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c" event={"ID":"f0d799e3-5a03-4723-84cc-c0e6e819ba9b","Type":"ContainerDied","Data":"b3b915f75d0df591ab3d1435f6c1e49ca478ed7b848f874ecc945b79ef9631e4"}
Nov 02 12:15:21.647486 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:21.647402    4942 scope.go:122] "RemoveContainer" containerID="38b5bd45aa96ca9a073a82167d57ba166fd4bfaf0a7bd64205a0c496b533639d"
Nov 02 12:15:21.815002 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:21.814923    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c"]
Nov 02 12:15:21.817983 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:21.817959    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w4-03c4e8-29893695-zmn8c"]
Nov 02 12:15:22.493016 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:22.492975    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 12:15:22.493393 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:22.493050    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk" podUID="5596075f-9595-476d-a6c2-45234fa26395" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 12:15:23.493199 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:23.493151    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 12:15:23.493729 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:23.493237    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk" podUID="5596075f-9595-476d-a6c2-45234fa26395" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 12:15:23.543291 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:23.543260    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0d799e3-5a03-4723-84cc-c0e6e819ba9b" path="/var/lib/kubelet/pods/f0d799e3-5a03-4723-84cc-c0e6e819ba9b/volumes"
Nov 02 12:15:24.492428 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:24.492397    4942 patch_prober.go:29] interesting pod/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 02 12:15:24.492608 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:24.492445    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk" podUID="5596075f-9595-476d-a6c2-45234fa26395" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 02 12:15:25.716165 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:25.716130    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:25.716611 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:25.716577    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:25.717343 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:25.717319    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:15:25.938258 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:25.938224    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"]
Nov 02 12:15:25.938778 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:25.938765    4942 cpu_manager.go:414] "containerMap: removing container" podUID="f0d799e3-5a03-4723-84cc-c0e6e819ba9b" containerName="cleanup"
Nov 02 12:15:25.938826 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:25.938779    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0d799e3-5a03-4723-84cc-c0e6e819ba9b" containerName="cleanup"
Nov 02 12:15:25.938826 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:25.938792    4942 cpu_manager.go:414] "containerMap: removing container" podUID="f0d799e3-5a03-4723-84cc-c0e6e819ba9b" containerName="cleanup"
Nov 02 12:15:25.938826 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:25.938797    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0d799e3-5a03-4723-84cc-c0e6e819ba9b" containerName="cleanup"
Nov 02 12:15:25.938826 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:25.938805    4942 cpu_manager.go:414] "containerMap: removing container" podUID="f0d799e3-5a03-4723-84cc-c0e6e819ba9b" containerName="cleanup"
Nov 02 12:15:25.938826 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:25.938811    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0d799e3-5a03-4723-84cc-c0e6e819ba9b" containerName="cleanup"
Nov 02 12:15:25.938998 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:25.938897    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0d799e3-5a03-4723-84cc-c0e6e819ba9b" containerName="cleanup"
Nov 02 12:15:25.938998 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:25.938906    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0d799e3-5a03-4723-84cc-c0e6e819ba9b" containerName="cleanup"
Nov 02 12:15:25.938998 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:25.938913    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0d799e3-5a03-4723-84cc-c0e6e819ba9b" containerName="cleanup"
Nov 02 12:15:25.948021 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:25.947979    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"
Nov 02 12:15:26.003606 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:26.003534    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-6ac03f-serving-cert\""
Nov 02 12:15:26.005145 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:26.005120    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"]
Nov 02 12:15:26.067322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:26.059792    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cf8ac58-6048-4b60-8f6a-487dde6cd593-maas-api-tls\") pod \"maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz\" (UID: \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"
Nov 02 12:15:26.067322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:26.059910    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrlvl\" (UniqueName: \"kubernetes.io/projected/3cf8ac58-6048-4b60-8f6a-487dde6cd593-kube-api-access-mrlvl\") pod \"maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz\" (UID: \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"
Nov 02 12:15:26.067322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:26.059961    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cf8ac58-6048-4b60-8f6a-487dde6cd593-metrics-certs\") pod \"maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz\" (UID: \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"
Nov 02 12:15:26.067322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:26.060000    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf8ac58-6048-4b60-8f6a-487dde6cd593-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz\" (UID: \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"
Nov 02 12:15:26.162452 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:26.162414    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cf8ac58-6048-4b60-8f6a-487dde6cd593-maas-api-tls\") pod \"maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz\" (UID: \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"
Nov 02 12:15:26.162633 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:26.162492    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrlvl\" (UniqueName: \"kubernetes.io/projected/3cf8ac58-6048-4b60-8f6a-487dde6cd593-kube-api-access-mrlvl\") pod \"maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz\" (UID: \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"
Nov 02 12:15:26.162633 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:26.162520    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cf8ac58-6048-4b60-8f6a-487dde6cd593-metrics-certs\") pod \"maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz\" (UID: \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"
Nov 02 12:15:26.162633 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:26.162543    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf8ac58-6048-4b60-8f6a-487dde6cd593-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz\" (UID: \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"
Nov 02 12:15:26.163131 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:26.163109    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf8ac58-6048-4b60-8f6a-487dde6cd593-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz\" (UID: \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"
Nov 02 12:15:26.165823 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:26.165797    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cf8ac58-6048-4b60-8f6a-487dde6cd593-metrics-certs\") pod \"maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz\" (UID: \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"
Nov 02 12:15:26.166606 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:26.166580    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cf8ac58-6048-4b60-8f6a-487dde6cd593-maas-api-tls\") pod \"maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz\" (UID: \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"
Nov 02 12:15:26.223910 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:26.223882    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrlvl\" (UniqueName: \"kubernetes.io/projected/3cf8ac58-6048-4b60-8f6a-487dde6cd593-kube-api-access-mrlvl\") pod \"maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz\" (UID: \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"
Nov 02 12:15:26.266988 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:26.266917    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"
Nov 02 12:15:26.645004 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:26.644973    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"]
Nov 02 12:15:26.645477 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:15:26.645450    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cf8ac58_6048_4b60_8f6a_487dde6cd593.slice/crio-89f2a3ecbe69df24b81a514a7b56f46461840d65d7f8335fe6001cb427a8a67c WatchSource:0}: Error finding container 89f2a3ecbe69df24b81a514a7b56f46461840d65d7f8335fe6001cb427a8a67c: Status 404 returned error can't find the container with id 89f2a3ecbe69df24b81a514a7b56f46461840d65d7f8335fe6001cb427a8a67c
Nov 02 12:15:26.707752 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:26.707716    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz" event={"ID":"3cf8ac58-6048-4b60-8f6a-487dde6cd593","Type":"ContainerStarted","Data":"89f2a3ecbe69df24b81a514a7b56f46461840d65d7f8335fe6001cb427a8a67c"}
Nov 02 12:15:27.717517 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:27.717480    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz" event={"ID":"3cf8ac58-6048-4b60-8f6a-487dde6cd593","Type":"ContainerStarted","Data":"9bfab9066dafa14a4ea2e57059aef1b3763d487a1710d46b8e8487429a69ba8e"}
Nov 02 12:15:27.717954 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:27.717548    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"
Nov 02 12:15:27.861619 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:27.861572    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz" podStartSLOduration=1.94424251 podStartE2EDuration="2.86155818s" totalImagesPullingTime="917.31567ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:15:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:15:27.860529212 +0000 UTC m=+1154.431878972" watchObservedRunningTime="2026-11-02 12:15:27.86155818 +0000 UTC m=+1154.432907948"
Nov 02 12:15:30.213429 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:30.213380    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8" podUID="e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 02 12:15:30.324323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:30.323406    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f" podUID="5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 12:15:30.685509 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:30.685445    4942 prober.go:122] "Probe failed" probeType="Liveness" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8" podUID="e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:15:31.161323 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:15:31.158330    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:15:31.214466 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:31.214395    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8" podUID="e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:15:31.215872 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:31.215849    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:15:31.305321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:31.304870    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:15:33.742975 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:33.742943    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"
Nov 02 12:15:44.450748 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.446217    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"]
Nov 02 12:15:44.480318 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.464055    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.523716 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.523681    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w4-03c4e8-openshift-default-dockercfg-8bvrq\""
Nov 02 12:15:44.654323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.634367    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"]
Nov 02 12:15:44.654323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.649010    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldcn9\" (UniqueName: \"kubernetes.io/projected/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-kube-api-access-ldcn9\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.654323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.649063    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-credential-socket\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.654323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.649099    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-workload-certs\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.654323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.649147    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-envoy\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.654323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.649214    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-token\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.654323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.649249    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-data\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.654323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.649315    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istiod-ca-cert\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.654323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.649445    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-workload-socket\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.654323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.649531    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-podinfo\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.767491 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.767366    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-workload-certs\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.783607 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.783581    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-envoy\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.787481 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.768129    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-workload-certs\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.791533 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.791508    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-token\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.791754 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.791724    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-data\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.799351 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.791914    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istiod-ca-cert\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.807365 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.803472    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-workload-socket\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.831225 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.824063    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istiod-ca-cert\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.831483 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.824324    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-data\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.831483 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.830731    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-workload-socket\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.832281 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.832170    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-podinfo\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.832691 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.832670    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldcn9\" (UniqueName: \"kubernetes.io/projected/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-kube-api-access-ldcn9\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.871242 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.836382    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-credential-socket\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.873514 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.873487    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-podinfo\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.885421 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.885392    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-credential-socket\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.960309 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.960259    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-envoy\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.970041 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.969976    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-token\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:44.986860 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:44.986801    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldcn9\" (UniqueName: \"kubernetes.io/projected/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-kube-api-access-ldcn9\") pod \"e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") " pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:45.091555 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:45.091469    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:46.195345 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:46.186180    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"]
Nov 02 12:15:46.320317 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:46.320253    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:15:46.327454 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:46.327408    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:15:46.339145 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:46.339106    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:15:47.130608 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:47.128567    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj" event={"ID":"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee","Type":"ContainerStarted","Data":"1061706a3d4f9ab55aa4630c9f7cefd585fa4d9d028980128888f517c82f3147"}
Nov 02 12:15:47.130608 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:47.128674    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj" event={"ID":"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee","Type":"ContainerStarted","Data":"30f0bc6ac48d157db9197cc39678d80a66aedd19ad82fd65c2bd065b832be125"}
Nov 02 12:15:47.368252 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:47.368188    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj" podStartSLOduration=3.368166363 podStartE2EDuration="3.368166363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:15:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:15:47.313606743 +0000 UTC m=+1173.884956504" watchObservedRunningTime="2026-11-02 12:15:47.368166363 +0000 UTC m=+1173.939516121"
Nov 02 12:15:48.092999 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:48.092960    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:48.094531 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:48.094499    4942 patch_prober.go:29] interesting pod/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 12:15:48.094686 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:48.094552    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj" podUID="76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 12:15:49.092808 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:49.092766    4942 patch_prober.go:29] interesting pod/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 12:15:49.093270 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:49.092828    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj" podUID="76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 12:15:50.092963 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:50.092928    4942 patch_prober.go:29] interesting pod/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 12:15:50.093370 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:50.092982    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj" podUID="76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 12:15:51.092479 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:51.092440    4942 patch_prober.go:29] interesting pod/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 12:15:51.092666 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:51.092520    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj" podUID="76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 12:15:52.092599 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:52.092563    4942 patch_prober.go:29] interesting pod/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 12:15:52.093010 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:52.092630    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj" podUID="76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 12:15:53.092709 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:53.092677    4942 patch_prober.go:29] interesting pod/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 02 12:15:53.093095 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:53.092727    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj" podUID="76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 02 12:15:54.096928 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:54.096894    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:54.097431 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:54.097416    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:15:54.098121 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:15:54.098097    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:16:00.186323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:00.185430    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw" podUID="242aa9d0-49d5-4981-ad74-0f7b95635e6a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 02 12:16:00.312321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:00.306252    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8" podUID="e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 02 12:16:00.344475 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:00.344431    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f" podUID="5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 12:16:00.984027 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:16:00.983944    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:16:10.023929 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:10.023895    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"]
Nov 02 12:16:13.703826 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:13.703795    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:16:13.709906 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:13.709879    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:16:17.332453 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:17.332397    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f" podUID="5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:16:25.043807 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.043724    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8" podUID="e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" containerName="main" containerID="cri-o://4caa2503921977a916800847bda03bac368ff45b776f391600879767bbb31990" gracePeriod=45
Nov 02 12:16:25.314641 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.314618    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:16:25.434386 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.434355    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-tmp-dir\") pod \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") "
Nov 02 12:16:25.434536 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.434410    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-dshm\") pod \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") "
Nov 02 12:16:25.434585 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.434536    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-home\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-home\") pod \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") "
Nov 02 12:16:25.434634 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.434586    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-model-cache\") pod \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") "
Nov 02 12:16:25.434634 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.434607    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-kube-api-access-ckd8m\" (UniqueName: \"kubernetes.io/projected/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-kube-api-access-ckd8m\") pod \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") "
Nov 02 12:16:25.434747 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.434624    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-tmp-dir" pod "e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" (UID: "e8d2041b-9e25-42f5-aee6-8e4ac3a441d2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:16:25.434747 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.434651    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-tls-certs\" (UniqueName: \"kubernetes.io/secret/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-tls-certs\") pod \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\" (UID: \"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2\") "
Nov 02 12:16:25.434873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.434848    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-model-cache" pod "e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" (UID: "e8d2041b-9e25-42f5-aee6-8e4ac3a441d2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:16:25.434938 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.434864    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-home" pod "e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" (UID: "e8d2041b-9e25-42f5-aee6-8e4ac3a441d2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:16:25.435106 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.435087    4942 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-home\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:16:25.435174 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.435113    4942 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-model-cache\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:16:25.435174 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.435129    4942 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-tmp-dir\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:16:25.437083 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.437051    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-kube-api-access-ckd8m" pod "e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" (UID: "e8d2041b-9e25-42f5-aee6-8e4ac3a441d2"). InnerVolumeSpecName "kube-api-access-ckd8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:16:25.437083 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.437075    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-tls-certs" pod "e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" (UID: "e8d2041b-9e25-42f5-aee6-8e4ac3a441d2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:16:25.437230 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.437095    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-dshm" pod "e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" (UID: "e8d2041b-9e25-42f5-aee6-8e4ac3a441d2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:16:25.531262 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.531234    4942 generic.go:356] "Generic (PLEG): container finished" podID="e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" containerID="4caa2503921977a916800847bda03bac368ff45b776f391600879767bbb31990" exitCode=0
Nov 02 12:16:25.531417 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.531273    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"
Nov 02 12:16:25.531417 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.531320    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8" event={"ID":"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2","Type":"ContainerDied","Data":"4caa2503921977a916800847bda03bac368ff45b776f391600879767bbb31990"}
Nov 02 12:16:25.531417 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.531363    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8" event={"ID":"e8d2041b-9e25-42f5-aee6-8e4ac3a441d2","Type":"ContainerDied","Data":"256f58f76e237e9f6b187d86629685afb374ec68e5f84cf980a7885f46180b1c"}
Nov 02 12:16:25.531417 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.531384    4942 scope.go:122] "RemoveContainer" containerID="4caa2503921977a916800847bda03bac368ff45b776f391600879767bbb31990"
Nov 02 12:16:25.536002 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.535933    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckd8m\" (UniqueName: \"kubernetes.io/projected/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-kube-api-access-ckd8m\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:16:25.536002 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.535966    4942 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-tls-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:16:25.536002 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.535981    4942 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2-dshm\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:16:25.542381 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.542366    4942 scope.go:122] "RemoveContainer" containerID="4caa2503921977a916800847bda03bac368ff45b776f391600879767bbb31990"
Nov 02 12:16:25.542637 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:16:25.542619    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4caa2503921977a916800847bda03bac368ff45b776f391600879767bbb31990\": container with ID starting with 4caa2503921977a916800847bda03bac368ff45b776f391600879767bbb31990 not found: ID does not exist" containerID="4caa2503921977a916800847bda03bac368ff45b776f391600879767bbb31990"
Nov 02 12:16:25.542696 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.542644    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4caa2503921977a916800847bda03bac368ff45b776f391600879767bbb31990"} err="failed to get container status \"4caa2503921977a916800847bda03bac368ff45b776f391600879767bbb31990\": rpc error: code = NotFound desc = could not find container \"4caa2503921977a916800847bda03bac368ff45b776f391600879767bbb31990\": container with ID starting with 4caa2503921977a916800847bda03bac368ff45b776f391600879767bbb31990 not found: ID does not exist"
Nov 02 12:16:25.695627 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.695596    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"]
Nov 02 12:16:25.700084 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:25.700059    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8"]
Nov 02 12:16:26.304742 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:26.304699    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w4-dd087a/auth-test-model-dd087a-kserve-8df5d595c-bgnk8" podUID="e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded"
Nov 02 12:16:27.547482 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:16:27.547447    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" path="/var/lib/kubelet/pods/e8d2041b-9e25-42f5-aee6-8e4ac3a441d2/volumes"
Nov 02 12:16:30.983105 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:16:30.983066    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:17:00.984600 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:17:00.984563    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:17:24.122208 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.122168    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"]
Nov 02 12:17:24.123111 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.123090    4942 cpu_manager.go:414] "containerMap: removing container" podUID="e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" containerName="main"
Nov 02 12:17:24.123167 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.123115    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" containerName="main"
Nov 02 12:17:24.123254 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.123242    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8d2041b-9e25-42f5-aee6-8e4ac3a441d2" containerName="main"
Nov 02 12:17:24.129628 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.129607    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.187153 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.187114    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"]
Nov 02 12:17:24.189054 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.189009    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbsxd\" (UniqueName: \"kubernetes.io/projected/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-kube-api-access-hbsxd\") pod \"distinct-w0-b1703c-kserve-7ff9784d57-vtchb\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") " pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.189319 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.189284    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-model-cache\") pod \"distinct-w0-b1703c-kserve-7ff9784d57-vtchb\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") " pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.189463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.189445    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-dshm\") pod \"distinct-w0-b1703c-kserve-7ff9784d57-vtchb\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") " pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.189537 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.189474    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-tls-certs\") pod \"distinct-w0-b1703c-kserve-7ff9784d57-vtchb\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") " pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.189537 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.189504    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-home\") pod \"distinct-w0-b1703c-kserve-7ff9784d57-vtchb\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") " pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.189647 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.189563    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-tmp-dir\") pod \"distinct-w0-b1703c-kserve-7ff9784d57-vtchb\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") " pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.231290 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.231257    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-b1703c\"/\"openshift-service-ca.crt\""
Nov 02 12:17:24.231290 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.231261    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-b1703c\"/\"default-dockercfg-ptjkv\""
Nov 02 12:17:24.237197 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.237171    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-b1703c\"/\"kube-root-ca.crt\""
Nov 02 12:17:24.237325 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.237240    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-b1703c\"/\"distinct-w0-b1703c-kserve-self-signed-certs\""
Nov 02 12:17:24.295671 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.292191    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-dshm\") pod \"distinct-w0-b1703c-kserve-7ff9784d57-vtchb\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") " pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.296219 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.296184    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-tls-certs\") pod \"distinct-w0-b1703c-kserve-7ff9784d57-vtchb\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") " pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.296441 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.296422    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-home\") pod \"distinct-w0-b1703c-kserve-7ff9784d57-vtchb\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") " pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.296595 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.296580    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-tmp-dir\") pod \"distinct-w0-b1703c-kserve-7ff9784d57-vtchb\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") " pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.296808 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.296790    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbsxd\" (UniqueName: \"kubernetes.io/projected/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-kube-api-access-hbsxd\") pod \"distinct-w0-b1703c-kserve-7ff9784d57-vtchb\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") " pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.296953 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.296938    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-model-cache\") pod \"distinct-w0-b1703c-kserve-7ff9784d57-vtchb\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") " pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.299846 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.299653    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-model-cache\") pod \"distinct-w0-b1703c-kserve-7ff9784d57-vtchb\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") " pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.300260 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.300239    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-tmp-dir\") pod \"distinct-w0-b1703c-kserve-7ff9784d57-vtchb\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") " pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.300792 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.300771    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-home\") pod \"distinct-w0-b1703c-kserve-7ff9784d57-vtchb\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") " pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.307256 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.307229    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-tls-certs\") pod \"distinct-w0-b1703c-kserve-7ff9784d57-vtchb\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") " pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.311268 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.311247    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-dshm\") pod \"distinct-w0-b1703c-kserve-7ff9784d57-vtchb\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") " pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.366942 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.366890    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbsxd\" (UniqueName: \"kubernetes.io/projected/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-kube-api-access-hbsxd\") pod \"distinct-w0-b1703c-kserve-7ff9784d57-vtchb\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") " pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.445570 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.445497    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:24.843924 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:24.843895    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"]
Nov 02 12:17:24.844552 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:17:24.844524    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca691fd2_312b_49cd_9bdd_4010c8fb10fc.slice/crio-49040151e9c4753c842bb5113a7cac6684f0d5221dc193b9e7b671e60744563f WatchSource:0}: Error finding container 49040151e9c4753c842bb5113a7cac6684f0d5221dc193b9e7b671e60744563f: Status 404 returned error can't find the container with id 49040151e9c4753c842bb5113a7cac6684f0d5221dc193b9e7b671e60744563f
Nov 02 12:17:25.208854 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:25.208818    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb" event={"ID":"ca691fd2-312b-49cd-9bdd-4010c8fb10fc","Type":"ContainerStarted","Data":"3747016f339933d6b9026fd1d2067dd64d8e62b3a3bd7c14d26e1bff6b66d3c2"}
Nov 02 12:17:25.209311 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:25.208864    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb" event={"ID":"ca691fd2-312b-49cd-9bdd-4010c8fb10fc","Type":"ContainerStarted","Data":"49040151e9c4753c842bb5113a7cac6684f0d5221dc193b9e7b671e60744563f"}
Nov 02 12:17:25.328085 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:25.328038    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb" podStartSLOduration=1.328023983 podStartE2EDuration="1.328023983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:17:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:17:25.326487461 +0000 UTC m=+1271.897837217" watchObservedRunningTime="2026-11-02 12:17:25.328023983 +0000 UTC m=+1271.899373740"
Nov 02 12:17:25.446239 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:25.446207    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:30.996408 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:17:30.996354    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:17:34.446248 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:34.446212    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:34.458535 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:34.458513    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:35.303267 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:35.303236    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:17:42.717323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:42.717242    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"]
Nov 02 12:17:42.733322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:42.730572    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:42.906321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:42.905580    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w4-03c4e8\"/\"kube-root-ca.crt\""
Nov 02 12:17:42.906321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:42.905699    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w4-03c4e8\"/\"openshift-service-ca.crt\""
Nov 02 12:17:42.906321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:42.905804    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w4-03c4e8\"/\"default-dockercfg-7bxqj\""
Nov 02 12:17:42.906321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:42.905900    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w4-03c4e8\"/\"rate-test-model-03c4e8-kserve-self-signed-certs\""
Nov 02 12:17:43.049536 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:42.977682    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-home\") pod \"rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") " pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:43.049846 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.049811    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-tmp-dir\") pod \"rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") " pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:43.058323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.053457    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ntd5\" (UniqueName: \"kubernetes.io/projected/f3d8189b-983b-488a-9419-788fdf6f93cb-kube-api-access-7ntd5\") pod \"rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") " pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:43.062320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.058587    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-model-cache\") pod \"rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") " pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:43.066440 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.066412    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3d8189b-983b-488a-9419-788fdf6f93cb-tls-certs\") pod \"rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") " pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:43.092322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.070356    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-dshm\") pod \"rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") " pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:43.092884 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.092852    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"]
Nov 02 12:17:43.197160 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.197114    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3d8189b-983b-488a-9419-788fdf6f93cb-tls-certs\") pod \"rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") " pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:43.198100 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.198066    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-dshm\") pod \"rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") " pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:43.199422 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.199346    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-home\") pod \"rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") " pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:43.199598 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.199579    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-home\") pod \"rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") " pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:43.200196 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.200139    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-tmp-dir\") pod \"rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") " pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:43.200374 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.200356    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-tmp-dir\") pod \"rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") " pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:43.211336 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.210983    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ntd5\" (UniqueName: \"kubernetes.io/projected/f3d8189b-983b-488a-9419-788fdf6f93cb-kube-api-access-7ntd5\") pod \"rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") " pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:43.211336 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.211101    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-model-cache\") pod \"rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") " pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:43.211828 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.211775    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-model-cache\") pod \"rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") " pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:43.213435 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.213397    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw" podUID="242aa9d0-49d5-4981-ad74-0f7b95635e6a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:17:43.266778 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.263616    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-dshm\") pod \"rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") " pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:43.288236 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.288160    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3d8189b-983b-488a-9419-788fdf6f93cb-tls-certs\") pod \"rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") " pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:43.289069 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.289043    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ntd5\" (UniqueName: \"kubernetes.io/projected/f3d8189b-983b-488a-9419-788fdf6f93cb-kube-api-access-7ntd5\") pod \"rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") " pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:43.470324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:43.469494    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:44.263548 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:44.260422    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw" podUID="242aa9d0-49d5-4981-ad74-0f7b95635e6a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:17:44.278321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:44.264031    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:17:44.302435 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:44.302385    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb" podUID="ca691fd2-312b-49cd-9bdd-4010c8fb10fc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:17:44.337515 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:44.336740    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f" podUID="5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:17:44.448543 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:44.448514    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:17:44.620502 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:17:44.620467    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3d8189b_983b_488a_9419_788fdf6f93cb.slice/crio-5ca6aeadca5f55b9ef65008e747aabeec2beabf8b11f1c63b904c32d0cc51fd8 WatchSource:0}: Error finding container 5ca6aeadca5f55b9ef65008e747aabeec2beabf8b11f1c63b904c32d0cc51fd8: Status 404 returned error can't find the container with id 5ca6aeadca5f55b9ef65008e747aabeec2beabf8b11f1c63b904c32d0cc51fd8
Nov 02 12:17:44.832405 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:44.820365    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"]
Nov 02 12:17:45.547089 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:45.547053    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28" event={"ID":"f3d8189b-983b-488a-9419-788fdf6f93cb","Type":"ContainerStarted","Data":"f274bf9d845df772d3bcf9718fc8362009a999572727c257f3e90d9e4c838a5f"}
Nov 02 12:17:45.547579 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:45.547103    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28" event={"ID":"f3d8189b-983b-488a-9419-788fdf6f93cb","Type":"ContainerStarted","Data":"5ca6aeadca5f55b9ef65008e747aabeec2beabf8b11f1c63b904c32d0cc51fd8"}
Nov 02 12:17:45.666071 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:45.666015    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28" podStartSLOduration=3.666000715 podStartE2EDuration="3.666000715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:17:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:17:45.662159225 +0000 UTC m=+1292.233508986" watchObservedRunningTime="2026-11-02 12:17:45.666000715 +0000 UTC m=+1292.237350471"
Nov 02 12:17:46.470175 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:46.470143    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:53.471154 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:53.471118    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:53.501924 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:53.501888    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:17:53.620280 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:17:53.620253    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:18:00.983423 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:18:00.983389    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:18:06.097664 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.097573    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"]
Nov 02 12:18:06.102652 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.102627    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"
Nov 02 12:18:06.155293 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.155119    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-3a78b4\"/\"agc-cred-14e0f93131b7\""
Nov 02 12:18:06.177563 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.177515    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0148a72c-1dde-4505-b63e-ab71dce0c1a7-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr\" (UID: \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"
Nov 02 12:18:06.177699 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.177569    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0148a72c-1dde-4505-b63e-ab71dce0c1a7-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr\" (UID: \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"
Nov 02 12:18:06.177741 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.177688    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr6w2\" (UniqueName: \"kubernetes.io/projected/0148a72c-1dde-4505-b63e-ab71dce0c1a7-kube-api-access-tr6w2\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr\" (UID: \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"
Nov 02 12:18:06.177898 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.177872    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0148a72c-1dde-4505-b63e-ab71dce0c1a7-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr\" (UID: \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"
Nov 02 12:18:06.204709 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.204675    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-3a78b4\"/\"openshift-service-ca.crt\""
Nov 02 12:18:06.204867 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.204735    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-3a78b4\"/\"kube-root-ca.crt\""
Nov 02 12:18:06.204867 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.204781    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-3a78b4\"/\"routing-overlay\""
Nov 02 12:18:06.204867 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.204792    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-3a78b4\"/\"payload-processing-external-model-plugins-e2e-extmod-b-3a78b4\""
Nov 02 12:18:06.205041 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.204882    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-3a78b4\"/\"payload-processing-external-model-e2e--a8ab13a3-dockercfg-wmcbx\""
Nov 02 12:18:06.208795 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.208769    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"]
Nov 02 12:18:06.279510 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.279482    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0148a72c-1dde-4505-b63e-ab71dce0c1a7-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr\" (UID: \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"
Nov 02 12:18:06.279652 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.279637    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0148a72c-1dde-4505-b63e-ab71dce0c1a7-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr\" (UID: \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"
Nov 02 12:18:06.279733 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.279659    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0148a72c-1dde-4505-b63e-ab71dce0c1a7-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr\" (UID: \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"
Nov 02 12:18:06.279733 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.279697    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr6w2\" (UniqueName: \"kubernetes.io/projected/0148a72c-1dde-4505-b63e-ab71dce0c1a7-kube-api-access-tr6w2\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr\" (UID: \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"
Nov 02 12:18:06.280183 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.280159    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0148a72c-1dde-4505-b63e-ab71dce0c1a7-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr\" (UID: \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"
Nov 02 12:18:06.280419 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.280401    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0148a72c-1dde-4505-b63e-ab71dce0c1a7-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr\" (UID: \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"
Nov 02 12:18:06.282014 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.281993    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0148a72c-1dde-4505-b63e-ab71dce0c1a7-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr\" (UID: \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"
Nov 02 12:18:06.340118 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.340094    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr6w2\" (UniqueName: \"kubernetes.io/projected/0148a72c-1dde-4505-b63e-ab71dce0c1a7-kube-api-access-tr6w2\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr\" (UID: \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"
Nov 02 12:18:06.419280 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.419247    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"
Nov 02 12:18:06.809019 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:06.808987    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"]
Nov 02 12:18:06.809436 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:18:06.809411    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0148a72c_1dde_4505_b63e_ab71dce0c1a7.slice/crio-fe84bc4bc6999ac4c139bf13ea07418241b9a1354548d7173e0c96bd6074f564 WatchSource:0}: Error finding container fe84bc4bc6999ac4c139bf13ea07418241b9a1354548d7173e0c96bd6074f564: Status 404 returned error can't find the container with id fe84bc4bc6999ac4c139bf13ea07418241b9a1354548d7173e0c96bd6074f564
Nov 02 12:18:07.713193 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:07.713158    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr" event={"ID":"0148a72c-1dde-4505-b63e-ab71dce0c1a7","Type":"ContainerStarted","Data":"7dd66494d33034214cddc5512f48aec647eac008d215cdfbdcf60f54338bf6f8"}
Nov 02 12:18:07.713611 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:07.713208    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr" event={"ID":"0148a72c-1dde-4505-b63e-ab71dce0c1a7","Type":"ContainerStarted","Data":"fe84bc4bc6999ac4c139bf13ea07418241b9a1354548d7173e0c96bd6074f564"}
Nov 02 12:18:07.713611 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:07.713250    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"
Nov 02 12:18:07.833443 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:07.833394    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr" podStartSLOduration=1.833380103 podStartE2EDuration="1.833380103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:18:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:18:07.830341694 +0000 UTC m=+1314.401691448" watchObservedRunningTime="2026-11-02 12:18:07.833380103 +0000 UTC m=+1314.404729860"
Nov 02 12:18:18.722979 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:18.722950    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"
Nov 02 12:18:30.983348 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:18:30.983278    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:18:39.083442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:39.083406    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k"]
Nov 02 12:18:39.194557 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:39.194522    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84"]
Nov 02 12:18:41.329666 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:41.329548    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f" podUID="5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:18:42.657479 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:42.657429    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28" podUID="f3d8189b-983b-488a-9419-788fdf6f93cb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:18:43.451253 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:43.451126    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw" podUID="242aa9d0-49d5-4981-ad74-0f7b95635e6a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 02 12:18:44.148106 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:44.145911    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k" podUID="fea24ab7-7d20-406c-9441-9fa3fdf93f61" containerName="payload-processing" containerID="cri-o://8766019d3f5d9ced89e908af4652ab25c2b839246e49ca097ad370ab239caa4c" gracePeriod=25
Nov 02 12:18:44.195144 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:44.195098    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84" podUID="04230821-2095-474d-aa72-8dc32b08ca0f" containerName="payload-pre-processing" containerID="cri-o://318e5e6c64043b07c9a42a7bfc4903d72a9ba9cdf0688364ef789635d9e94ac6" gracePeriod=25
Nov 02 12:18:53.558925 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:53.558879    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k" podUID="fea24ab7-7d20-406c-9441-9fa3fdf93f61" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 12:18:53.559354 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:18:53.558936    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84" podUID="04230821-2095-474d-aa72-8dc32b08ca0f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 12:19:01.329323 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:19:01.315484    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:19:01.453643 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:01.453586    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw" podUID="242aa9d0-49d5-4981-ad74-0f7b95635e6a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:19:02.296323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:02.295842    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb" podUID="ca691fd2-312b-49cd-9bdd-4010c8fb10fc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:19:02.337439 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:02.337376    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f" podUID="5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:19:02.458470 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:02.458426    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw" podUID="242aa9d0-49d5-4981-ad74-0f7b95635e6a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:19:02.459491 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:02.459466    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:19:02.615508 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:02.615468    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28" podUID="f3d8189b-983b-488a-9419-788fdf6f93cb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": context deadline exceeded"
Nov 02 12:19:03.125636 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:03.124941    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw"
Nov 02 12:19:03.559359 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:03.559262    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84" podUID="04230821-2095-474d-aa72-8dc32b08ca0f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 12:19:03.559726 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:03.559262    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k" podUID="fea24ab7-7d20-406c-9441-9fa3fdf93f61" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 12:19:04.422648 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:04.422558    4942 generic.go:356] "Generic (PLEG): container finished" podID="fea24ab7-7d20-406c-9441-9fa3fdf93f61" containerID="8766019d3f5d9ced89e908af4652ab25c2b839246e49ca097ad370ab239caa4c" exitCode=0
Nov 02 12:19:04.422802 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:04.422782    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k" event={"ID":"fea24ab7-7d20-406c-9441-9fa3fdf93f61","Type":"ContainerDied","Data":"8766019d3f5d9ced89e908af4652ab25c2b839246e49ca097ad370ab239caa4c"}
Nov 02 12:19:04.425074 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:04.425048    4942 generic.go:356] "Generic (PLEG): container finished" podID="04230821-2095-474d-aa72-8dc32b08ca0f" containerID="318e5e6c64043b07c9a42a7bfc4903d72a9ba9cdf0688364ef789635d9e94ac6" exitCode=0
Nov 02 12:19:04.425214 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:04.425120    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84" event={"ID":"04230821-2095-474d-aa72-8dc32b08ca0f","Type":"ContainerDied","Data":"318e5e6c64043b07c9a42a7bfc4903d72a9ba9cdf0688364ef789635d9e94ac6"}
Nov 02 12:19:04.483555 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:04.483531    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k"
Nov 02 12:19:04.520699 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:04.520675    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84"
Nov 02 12:19:04.561953 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:04.561926    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fea24ab7-7d20-406c-9441-9fa3fdf93f61-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fea24ab7-7d20-406c-9441-9fa3fdf93f61-plugins-config-volume\") pod \"fea24ab7-7d20-406c-9441-9fa3fdf93f61\" (UID: \"fea24ab7-7d20-406c-9441-9fa3fdf93f61\") "
Nov 02 12:19:04.562272 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:04.561965    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04230821-2095-474d-aa72-8dc32b08ca0f-kube-api-access-tsh8b\" (UniqueName: \"kubernetes.io/projected/04230821-2095-474d-aa72-8dc32b08ca0f-kube-api-access-tsh8b\") pod \"04230821-2095-474d-aa72-8dc32b08ca0f\" (UID: \"04230821-2095-474d-aa72-8dc32b08ca0f\") "
Nov 02 12:19:04.562272 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:04.562018    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/04230821-2095-474d-aa72-8dc32b08ca0f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04230821-2095-474d-aa72-8dc32b08ca0f-plugins-config-volume\") pod \"04230821-2095-474d-aa72-8dc32b08ca0f\" (UID: \"04230821-2095-474d-aa72-8dc32b08ca0f\") "
Nov 02 12:19:04.562272 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:04.562061    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fea24ab7-7d20-406c-9441-9fa3fdf93f61-kube-api-access-clrdf\" (UniqueName: \"kubernetes.io/projected/fea24ab7-7d20-406c-9441-9fa3fdf93f61-kube-api-access-clrdf\") pod \"fea24ab7-7d20-406c-9441-9fa3fdf93f61\" (UID: \"fea24ab7-7d20-406c-9441-9fa3fdf93f61\") "
Nov 02 12:19:04.562477 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:04.562358    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fea24ab7-7d20-406c-9441-9fa3fdf93f61-plugins-config-volume" pod "fea24ab7-7d20-406c-9441-9fa3fdf93f61" (UID: "fea24ab7-7d20-406c-9441-9fa3fdf93f61"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:19:04.562477 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:04.562411    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04230821-2095-474d-aa72-8dc32b08ca0f-plugins-config-volume" pod "04230821-2095-474d-aa72-8dc32b08ca0f" (UID: "04230821-2095-474d-aa72-8dc32b08ca0f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:19:04.562594 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:04.562514    4942 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/04230821-2095-474d-aa72-8dc32b08ca0f-plugins-config-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:04.562594 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:04.562534    4942 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fea24ab7-7d20-406c-9441-9fa3fdf93f61-plugins-config-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:04.564207 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:04.564186    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04230821-2095-474d-aa72-8dc32b08ca0f-kube-api-access-tsh8b" pod "04230821-2095-474d-aa72-8dc32b08ca0f" (UID: "04230821-2095-474d-aa72-8dc32b08ca0f"). InnerVolumeSpecName "kube-api-access-tsh8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:19:04.564292 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:04.564226    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fea24ab7-7d20-406c-9441-9fa3fdf93f61-kube-api-access-clrdf" pod "fea24ab7-7d20-406c-9441-9fa3fdf93f61" (UID: "fea24ab7-7d20-406c-9441-9fa3fdf93f61"). InnerVolumeSpecName "kube-api-access-clrdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:19:04.663961 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:04.663934    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsh8b\" (UniqueName: \"kubernetes.io/projected/04230821-2095-474d-aa72-8dc32b08ca0f-kube-api-access-tsh8b\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:04.663961 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:04.663960    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clrdf\" (UniqueName: \"kubernetes.io/projected/fea24ab7-7d20-406c-9441-9fa3fdf93f61-kube-api-access-clrdf\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:05.433544 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:05.433463    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k"
Nov 02 12:19:05.433544 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:05.433493    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k" event={"ID":"fea24ab7-7d20-406c-9441-9fa3fdf93f61","Type":"ContainerDied","Data":"a8f630d8793f44967bcde8555794d0ff36486f00f31cbea47a702c73b3ae4f14"}
Nov 02 12:19:05.433766 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:05.433559    4942 scope.go:122] "RemoveContainer" containerID="8766019d3f5d9ced89e908af4652ab25c2b839246e49ca097ad370ab239caa4c"
Nov 02 12:19:05.435048 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:05.435030    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84"
Nov 02 12:19:05.435132 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:05.435049    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84" event={"ID":"04230821-2095-474d-aa72-8dc32b08ca0f","Type":"ContainerDied","Data":"763836ad7be9243c48ccda494302c94d3feacb8ba547c65792969178d2eb652d"}
Nov 02 12:19:05.444879 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:05.444787    4942 scope.go:122] "RemoveContainer" containerID="318e5e6c64043b07c9a42a7bfc4903d72a9ba9cdf0688364ef789635d9e94ac6"
Nov 02 12:19:05.599082 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:05.599054    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k"]
Nov 02 12:19:05.648581 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:05.648549    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-7bd72e-698968c8bc-dps8k"]
Nov 02 12:19:05.717529 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:05.717443    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84"]
Nov 02 12:19:05.721759 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:05.721731    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-7bd72e-65db86f979-chl84"]
Nov 02 12:19:07.542832 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:07.542799    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="04230821-2095-474d-aa72-8dc32b08ca0f" path="/var/lib/kubelet/pods/04230821-2095-474d-aa72-8dc32b08ca0f/volumes"
Nov 02 12:19:07.543210 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:07.543199    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fea24ab7-7d20-406c-9441-9fa3fdf93f61" path="/var/lib/kubelet/pods/fea24ab7-7d20-406c-9441-9fa3fdf93f61/volumes"
Nov 02 12:19:12.147155 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.147120    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"]
Nov 02 12:19:12.147711 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.147349    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x" podUID="d6ec5343-b4bf-4508-819a-bb348b6dd0aa" containerName="maas-api" containerID="cri-o://1b0c2246f490b639dc3a118fa150545dfdce0a3833350edd128d03441f1a1309" gracePeriod=30
Nov 02 12:19:12.434645 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.434618    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:19:12.493682 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.493643    4942 generic.go:356] "Generic (PLEG): container finished" podID="d6ec5343-b4bf-4508-819a-bb348b6dd0aa" containerID="1b0c2246f490b639dc3a118fa150545dfdce0a3833350edd128d03441f1a1309" exitCode=0
Nov 02 12:19:12.493851 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.493695    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"
Nov 02 12:19:12.493851 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.493684    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x" event={"ID":"d6ec5343-b4bf-4508-819a-bb348b6dd0aa","Type":"ContainerDied","Data":"1b0c2246f490b639dc3a118fa150545dfdce0a3833350edd128d03441f1a1309"}
Nov 02 12:19:12.493851 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.493766    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x" event={"ID":"d6ec5343-b4bf-4508-819a-bb348b6dd0aa","Type":"ContainerDied","Data":"2c149a97d8020782501c5b20afb8d7cda1a683444f24cd4e78f660894ffb6478"}
Nov 02 12:19:12.493851 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.493798    4942 scope.go:122] "RemoveContainer" containerID="1b0c2246f490b639dc3a118fa150545dfdce0a3833350edd128d03441f1a1309"
Nov 02 12:19:12.503483 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.503462    4942 scope.go:122] "RemoveContainer" containerID="1b0c2246f490b639dc3a118fa150545dfdce0a3833350edd128d03441f1a1309"
Nov 02 12:19:12.503756 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:19:12.503737    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b0c2246f490b639dc3a118fa150545dfdce0a3833350edd128d03441f1a1309\": container with ID starting with 1b0c2246f490b639dc3a118fa150545dfdce0a3833350edd128d03441f1a1309 not found: ID does not exist" containerID="1b0c2246f490b639dc3a118fa150545dfdce0a3833350edd128d03441f1a1309"
Nov 02 12:19:12.503838 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.503768    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b0c2246f490b639dc3a118fa150545dfdce0a3833350edd128d03441f1a1309"} err="failed to get container status \"1b0c2246f490b639dc3a118fa150545dfdce0a3833350edd128d03441f1a1309\": rpc error: code = NotFound desc = could not find container \"1b0c2246f490b639dc3a118fa150545dfdce0a3833350edd128d03441f1a1309\": container with ID starting with 1b0c2246f490b639dc3a118fa150545dfdce0a3833350edd128d03441f1a1309 not found: ID does not exist"
Nov 02 12:19:12.551634 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.551610    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-metrics-certs\") pod \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\" (UID: \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\") "
Nov 02 12:19:12.551852 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.551727    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-maas-api-tls\") pod \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\" (UID: \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\") "
Nov 02 12:19:12.551852 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.551824    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-kube-api-access-nslkz\" (UniqueName: \"kubernetes.io/projected/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-kube-api-access-nslkz\") pod \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\" (UID: \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\") "
Nov 02 12:19:12.551975 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.551914    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-openshift-service-ca\") pod \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\" (UID: \"d6ec5343-b4bf-4508-819a-bb348b6dd0aa\") "
Nov 02 12:19:12.552711 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.552690    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-openshift-service-ca" pod "d6ec5343-b4bf-4508-819a-bb348b6dd0aa" (UID: "d6ec5343-b4bf-4508-819a-bb348b6dd0aa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:19:12.555114 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.554466    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-maas-api-tls" pod "d6ec5343-b4bf-4508-819a-bb348b6dd0aa" (UID: "d6ec5343-b4bf-4508-819a-bb348b6dd0aa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:19:12.555416 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.555387    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-kube-api-access-nslkz" pod "d6ec5343-b4bf-4508-819a-bb348b6dd0aa" (UID: "d6ec5343-b4bf-4508-819a-bb348b6dd0aa"). InnerVolumeSpecName "kube-api-access-nslkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:19:12.555502 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.555414    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-metrics-certs" pod "d6ec5343-b4bf-4508-819a-bb348b6dd0aa" (UID: "d6ec5343-b4bf-4508-819a-bb348b6dd0aa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:19:12.654034 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.654000    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:12.654034 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.654030    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:12.654034 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.654041    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:12.654311 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.654049    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nslkz\" (UniqueName: \"kubernetes.io/projected/d6ec5343-b4bf-4508-819a-bb348b6dd0aa-kube-api-access-nslkz\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:12.923356 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.923321    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"]
Nov 02 12:19:12.929520 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:12.929486    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7bd72e-7887cc77b5-z8t6x"]
Nov 02 12:19:13.554760 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:13.554727    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6ec5343-b4bf-4508-819a-bb348b6dd0aa" path="/var/lib/kubelet/pods/d6ec5343-b4bf-4508-819a-bb348b6dd0aa/volumes"
Nov 02 12:19:23.092318 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:23.092266    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"]
Nov 02 12:19:28.093080 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.092977    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr" podUID="0148a72c-1dde-4505-b63e-ab71dce0c1a7" containerName="payload-processing" containerID="cri-o://7dd66494d33034214cddc5512f48aec647eac008d215cdfbdcf60f54338bf6f8" gracePeriod=25
Nov 02 12:19:28.373215 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.373187    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"
Nov 02 12:19:28.398867 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.398835    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0148a72c-1dde-4505-b63e-ab71dce0c1a7-kube-api-access-tr6w2\" (UniqueName: \"kubernetes.io/projected/0148a72c-1dde-4505-b63e-ab71dce0c1a7-kube-api-access-tr6w2\") pod \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\" (UID: \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\") "
Nov 02 12:19:28.399023 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.398886    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0148a72c-1dde-4505-b63e-ab71dce0c1a7-provider-credentials\" (UniqueName: \"kubernetes.io/projected/0148a72c-1dde-4505-b63e-ab71dce0c1a7-provider-credentials\") pod \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\" (UID: \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\") "
Nov 02 12:19:28.399023 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.398942    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0148a72c-1dde-4505-b63e-ab71dce0c1a7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0148a72c-1dde-4505-b63e-ab71dce0c1a7-plugins-config-volume\") pod \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\" (UID: \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\") "
Nov 02 12:19:28.399023 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.398971    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0148a72c-1dde-4505-b63e-ab71dce0c1a7-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0148a72c-1dde-4505-b63e-ab71dce0c1a7-routing-overlay-volume\") pod \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\" (UID: \"0148a72c-1dde-4505-b63e-ab71dce0c1a7\") "
Nov 02 12:19:28.399649 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.399606    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0148a72c-1dde-4505-b63e-ab71dce0c1a7-plugins-config-volume" pod "0148a72c-1dde-4505-b63e-ab71dce0c1a7" (UID: "0148a72c-1dde-4505-b63e-ab71dce0c1a7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:19:28.399810 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.399678    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0148a72c-1dde-4505-b63e-ab71dce0c1a7-routing-overlay-volume" pod "0148a72c-1dde-4505-b63e-ab71dce0c1a7" (UID: "0148a72c-1dde-4505-b63e-ab71dce0c1a7"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:19:28.401509 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.401482    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0148a72c-1dde-4505-b63e-ab71dce0c1a7-kube-api-access-tr6w2" pod "0148a72c-1dde-4505-b63e-ab71dce0c1a7" (UID: "0148a72c-1dde-4505-b63e-ab71dce0c1a7"). InnerVolumeSpecName "kube-api-access-tr6w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:19:28.401614 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.401547    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0148a72c-1dde-4505-b63e-ab71dce0c1a7-provider-credentials" pod "0148a72c-1dde-4505-b63e-ab71dce0c1a7" (UID: "0148a72c-1dde-4505-b63e-ab71dce0c1a7"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:19:28.500937 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.500898    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tr6w2\" (UniqueName: \"kubernetes.io/projected/0148a72c-1dde-4505-b63e-ab71dce0c1a7-kube-api-access-tr6w2\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:28.500937 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.500940    4942 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0148a72c-1dde-4505-b63e-ab71dce0c1a7-provider-credentials\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:28.501125 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.500956    4942 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0148a72c-1dde-4505-b63e-ab71dce0c1a7-plugins-config-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:28.501125 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.500970    4942 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0148a72c-1dde-4505-b63e-ab71dce0c1a7-routing-overlay-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:28.641449 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.641413    4942 generic.go:356] "Generic (PLEG): container finished" podID="0148a72c-1dde-4505-b63e-ab71dce0c1a7" containerID="7dd66494d33034214cddc5512f48aec647eac008d215cdfbdcf60f54338bf6f8" exitCode=0
Nov 02 12:19:28.641642 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.641462    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"
Nov 02 12:19:28.641642 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.641491    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr" event={"ID":"0148a72c-1dde-4505-b63e-ab71dce0c1a7","Type":"ContainerDied","Data":"7dd66494d33034214cddc5512f48aec647eac008d215cdfbdcf60f54338bf6f8"}
Nov 02 12:19:28.641642 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.641541    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr" event={"ID":"0148a72c-1dde-4505-b63e-ab71dce0c1a7","Type":"ContainerDied","Data":"fe84bc4bc6999ac4c139bf13ea07418241b9a1354548d7173e0c96bd6074f564"}
Nov 02 12:19:28.641642 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.641569    4942 scope.go:122] "RemoveContainer" containerID="7dd66494d33034214cddc5512f48aec647eac008d215cdfbdcf60f54338bf6f8"
Nov 02 12:19:28.651744 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.651728    4942 scope.go:122] "RemoveContainer" containerID="7dd66494d33034214cddc5512f48aec647eac008d215cdfbdcf60f54338bf6f8"
Nov 02 12:19:28.652001 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:19:28.651983    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dd66494d33034214cddc5512f48aec647eac008d215cdfbdcf60f54338bf6f8\": container with ID starting with 7dd66494d33034214cddc5512f48aec647eac008d215cdfbdcf60f54338bf6f8 not found: ID does not exist" containerID="7dd66494d33034214cddc5512f48aec647eac008d215cdfbdcf60f54338bf6f8"
Nov 02 12:19:28.652049 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.652010    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dd66494d33034214cddc5512f48aec647eac008d215cdfbdcf60f54338bf6f8"} err="failed to get container status \"7dd66494d33034214cddc5512f48aec647eac008d215cdfbdcf60f54338bf6f8\": rpc error: code = NotFound desc = could not find container \"7dd66494d33034214cddc5512f48aec647eac008d215cdfbdcf60f54338bf6f8\": container with ID starting with 7dd66494d33034214cddc5512f48aec647eac008d215cdfbdcf60f54338bf6f8 not found: ID does not exist"
Nov 02 12:19:28.807791 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.807742    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"]
Nov 02 12:19:28.811658 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.811633    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-54d8tk5tr"]
Nov 02 12:19:28.995175 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.995090    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"]
Nov 02 12:19:28.995348 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:28.995307    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f" podUID="958b6369-5a4b-4a87-832e-66bc5e2db790" containerName="maas-api" containerID="cri-o://03193fdb5c613ba8f9571a7ee0f99b2e14edb7c0a6ac7daed8e8edb426d4d515" gracePeriod=30
Nov 02 12:19:29.362747 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.362711    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:19:29.414882 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.414850    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/958b6369-5a4b-4a87-832e-66bc5e2db790-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/958b6369-5a4b-4a87-832e-66bc5e2db790-maas-api-tls\") pod \"958b6369-5a4b-4a87-832e-66bc5e2db790\" (UID: \"958b6369-5a4b-4a87-832e-66bc5e2db790\") "
Nov 02 12:19:29.415084 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.414944    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/958b6369-5a4b-4a87-832e-66bc5e2db790-kube-api-access-z4r92\" (UniqueName: \"kubernetes.io/projected/958b6369-5a4b-4a87-832e-66bc5e2db790-kube-api-access-z4r92\") pod \"958b6369-5a4b-4a87-832e-66bc5e2db790\" (UID: \"958b6369-5a4b-4a87-832e-66bc5e2db790\") "
Nov 02 12:19:29.415084 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.415005    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/958b6369-5a4b-4a87-832e-66bc5e2db790-metrics-certs\" (UniqueName: \"kubernetes.io/secret/958b6369-5a4b-4a87-832e-66bc5e2db790-metrics-certs\") pod \"958b6369-5a4b-4a87-832e-66bc5e2db790\" (UID: \"958b6369-5a4b-4a87-832e-66bc5e2db790\") "
Nov 02 12:19:29.415084 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.415052    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/958b6369-5a4b-4a87-832e-66bc5e2db790-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/958b6369-5a4b-4a87-832e-66bc5e2db790-openshift-service-ca\") pod \"958b6369-5a4b-4a87-832e-66bc5e2db790\" (UID: \"958b6369-5a4b-4a87-832e-66bc5e2db790\") "
Nov 02 12:19:29.415787 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.415748    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/958b6369-5a4b-4a87-832e-66bc5e2db790-openshift-service-ca" pod "958b6369-5a4b-4a87-832e-66bc5e2db790" (UID: "958b6369-5a4b-4a87-832e-66bc5e2db790"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:19:29.417225 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.417163    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/958b6369-5a4b-4a87-832e-66bc5e2db790-maas-api-tls" pod "958b6369-5a4b-4a87-832e-66bc5e2db790" (UID: "958b6369-5a4b-4a87-832e-66bc5e2db790"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:19:29.417225 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.417187    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/958b6369-5a4b-4a87-832e-66bc5e2db790-metrics-certs" pod "958b6369-5a4b-4a87-832e-66bc5e2db790" (UID: "958b6369-5a4b-4a87-832e-66bc5e2db790"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:19:29.417442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.417334    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/958b6369-5a4b-4a87-832e-66bc5e2db790-kube-api-access-z4r92" pod "958b6369-5a4b-4a87-832e-66bc5e2db790" (UID: "958b6369-5a4b-4a87-832e-66bc5e2db790"). InnerVolumeSpecName "kube-api-access-z4r92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:19:29.523186 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.523145    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/958b6369-5a4b-4a87-832e-66bc5e2db790-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:29.523186 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.523187    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4r92\" (UniqueName: \"kubernetes.io/projected/958b6369-5a4b-4a87-832e-66bc5e2db790-kube-api-access-z4r92\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:29.523468 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.523203    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/958b6369-5a4b-4a87-832e-66bc5e2db790-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:29.523468 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.523218    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/958b6369-5a4b-4a87-832e-66bc5e2db790-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:29.544191 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.544160    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0148a72c-1dde-4505-b63e-ab71dce0c1a7" path="/var/lib/kubelet/pods/0148a72c-1dde-4505-b63e-ab71dce0c1a7/volumes"
Nov 02 12:19:29.651511 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.651477    4942 generic.go:356] "Generic (PLEG): container finished" podID="958b6369-5a4b-4a87-832e-66bc5e2db790" containerID="03193fdb5c613ba8f9571a7ee0f99b2e14edb7c0a6ac7daed8e8edb426d4d515" exitCode=0
Nov 02 12:19:29.651710 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.651518    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"
Nov 02 12:19:29.651710 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.651554    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f" event={"ID":"958b6369-5a4b-4a87-832e-66bc5e2db790","Type":"ContainerDied","Data":"03193fdb5c613ba8f9571a7ee0f99b2e14edb7c0a6ac7daed8e8edb426d4d515"}
Nov 02 12:19:29.651710 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.651620    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f" event={"ID":"958b6369-5a4b-4a87-832e-66bc5e2db790","Type":"ContainerDied","Data":"97afdab67f3b56cb8d59beb8cc8b6d92ce32531c562942d1a77ce811ac3b0bbb"}
Nov 02 12:19:29.651710 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.651650    4942 scope.go:122] "RemoveContainer" containerID="03193fdb5c613ba8f9571a7ee0f99b2e14edb7c0a6ac7daed8e8edb426d4d515"
Nov 02 12:19:29.663790 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.663766    4942 scope.go:122] "RemoveContainer" containerID="03193fdb5c613ba8f9571a7ee0f99b2e14edb7c0a6ac7daed8e8edb426d4d515"
Nov 02 12:19:29.664092 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:19:29.664067    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03193fdb5c613ba8f9571a7ee0f99b2e14edb7c0a6ac7daed8e8edb426d4d515\": container with ID starting with 03193fdb5c613ba8f9571a7ee0f99b2e14edb7c0a6ac7daed8e8edb426d4d515 not found: ID does not exist" containerID="03193fdb5c613ba8f9571a7ee0f99b2e14edb7c0a6ac7daed8e8edb426d4d515"
Nov 02 12:19:29.664173 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.664104    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03193fdb5c613ba8f9571a7ee0f99b2e14edb7c0a6ac7daed8e8edb426d4d515"} err="failed to get container status \"03193fdb5c613ba8f9571a7ee0f99b2e14edb7c0a6ac7daed8e8edb426d4d515\": rpc error: code = NotFound desc = could not find container \"03193fdb5c613ba8f9571a7ee0f99b2e14edb7c0a6ac7daed8e8edb426d4d515\": container with ID starting with 03193fdb5c613ba8f9571a7ee0f99b2e14edb7c0a6ac7daed8e8edb426d4d515 not found: ID does not exist"
Nov 02 12:19:29.819197 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.819165    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"]
Nov 02 12:19:29.823350 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:29.823326    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-036f88-5c556df6fd-jn67f"]
Nov 02 12:19:30.996254 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:19:30.996215    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:19:31.548763 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:31.548728    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="958b6369-5a4b-4a87-832e-66bc5e2db790" path="/var/lib/kubelet/pods/958b6369-5a4b-4a87-832e-66bc5e2db790/volumes"
Nov 02 12:19:40.332245 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.332209    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"]
Nov 02 12:19:40.332763 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.332748    4942 cpu_manager.go:414] "containerMap: removing container" podUID="0148a72c-1dde-4505-b63e-ab71dce0c1a7" containerName="payload-processing"
Nov 02 12:19:40.332810 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.332766    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0148a72c-1dde-4505-b63e-ab71dce0c1a7" containerName="payload-processing"
Nov 02 12:19:40.332810 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.332797    4942 cpu_manager.go:414] "containerMap: removing container" podUID="fea24ab7-7d20-406c-9441-9fa3fdf93f61" containerName="payload-processing"
Nov 02 12:19:40.332810 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.332802    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fea24ab7-7d20-406c-9441-9fa3fdf93f61" containerName="payload-processing"
Nov 02 12:19:40.332913 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.332825    4942 cpu_manager.go:414] "containerMap: removing container" podUID="04230821-2095-474d-aa72-8dc32b08ca0f" containerName="payload-pre-processing"
Nov 02 12:19:40.332913 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.332831    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04230821-2095-474d-aa72-8dc32b08ca0f" containerName="payload-pre-processing"
Nov 02 12:19:40.332913 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.332837    4942 cpu_manager.go:414] "containerMap: removing container" podUID="d6ec5343-b4bf-4508-819a-bb348b6dd0aa" containerName="maas-api"
Nov 02 12:19:40.332913 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.332841    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6ec5343-b4bf-4508-819a-bb348b6dd0aa" containerName="maas-api"
Nov 02 12:19:40.332913 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.332848    4942 cpu_manager.go:414] "containerMap: removing container" podUID="958b6369-5a4b-4a87-832e-66bc5e2db790" containerName="maas-api"
Nov 02 12:19:40.332913 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.332853    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="958b6369-5a4b-4a87-832e-66bc5e2db790" containerName="maas-api"
Nov 02 12:19:40.333092 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.332939    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="fea24ab7-7d20-406c-9441-9fa3fdf93f61" containerName="payload-processing"
Nov 02 12:19:40.333092 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.332950    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="04230821-2095-474d-aa72-8dc32b08ca0f" containerName="payload-pre-processing"
Nov 02 12:19:40.333092 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.332957    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6ec5343-b4bf-4508-819a-bb348b6dd0aa" containerName="maas-api"
Nov 02 12:19:40.333092 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.332962    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="958b6369-5a4b-4a87-832e-66bc5e2db790" containerName="maas-api"
Nov 02 12:19:40.333092 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.332969    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="0148a72c-1dde-4505-b63e-ab71dce0c1a7" containerName="payload-processing"
Nov 02 12:19:40.336358 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.336342    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"
Nov 02 12:19:40.390041 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.390017    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-3a78b4\"/\"agc-cred-50db61f22b2e\""
Nov 02 12:19:40.430766 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.430736    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/18798180-d3f0-4a87-8226-1b66d45a76bc-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2\" (UID: \"18798180-d3f0-4a87-8226-1b66d45a76bc\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"
Nov 02 12:19:40.430918 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.430801    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18798180-d3f0-4a87-8226-1b66d45a76bc-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2\" (UID: \"18798180-d3f0-4a87-8226-1b66d45a76bc\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"
Nov 02 12:19:40.430918 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.430899    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/18798180-d3f0-4a87-8226-1b66d45a76bc-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2\" (UID: \"18798180-d3f0-4a87-8226-1b66d45a76bc\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"
Nov 02 12:19:40.430997 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.430943    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwtnc\" (UniqueName: \"kubernetes.io/projected/18798180-d3f0-4a87-8226-1b66d45a76bc-kube-api-access-mwtnc\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2\" (UID: \"18798180-d3f0-4a87-8226-1b66d45a76bc\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"
Nov 02 12:19:40.439642 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.439617    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-3a78b4\"/\"openshift-service-ca.crt\""
Nov 02 12:19:40.439642 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.439617    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-3a78b4\"/\"agc-cred-14e0f93131b7\""
Nov 02 12:19:40.439642 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.439645    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-3a78b4\"/\"routing-overlay\""
Nov 02 12:19:40.439642 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.439622    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-3a78b4\"/\"kube-root-ca.crt\""
Nov 02 12:19:40.439935 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.439670    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-3a78b4\"/\"agc-cred-4b8a830d91a3\""
Nov 02 12:19:40.439935 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.439694    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-3a78b4\"/\"payload-processing-external-model-e2e--a8ab13a3-dockercfg-wmcbx\""
Nov 02 12:19:40.439935 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.439869    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-3a78b4\"/\"payload-processing-external-model-plugins-e2e-extmod-b-3a78b4\""
Nov 02 12:19:40.443263 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.443242    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"]
Nov 02 12:19:40.531853 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.531819    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/18798180-d3f0-4a87-8226-1b66d45a76bc-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2\" (UID: \"18798180-d3f0-4a87-8226-1b66d45a76bc\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"
Nov 02 12:19:40.531998 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.531920    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18798180-d3f0-4a87-8226-1b66d45a76bc-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2\" (UID: \"18798180-d3f0-4a87-8226-1b66d45a76bc\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"
Nov 02 12:19:40.531998 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.531966    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/18798180-d3f0-4a87-8226-1b66d45a76bc-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2\" (UID: \"18798180-d3f0-4a87-8226-1b66d45a76bc\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"
Nov 02 12:19:40.532065 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.532007    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwtnc\" (UniqueName: \"kubernetes.io/projected/18798180-d3f0-4a87-8226-1b66d45a76bc-kube-api-access-mwtnc\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2\" (UID: \"18798180-d3f0-4a87-8226-1b66d45a76bc\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"
Nov 02 12:19:40.532729 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.532702    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/18798180-d3f0-4a87-8226-1b66d45a76bc-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2\" (UID: \"18798180-d3f0-4a87-8226-1b66d45a76bc\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"
Nov 02 12:19:40.532834 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.532786    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18798180-d3f0-4a87-8226-1b66d45a76bc-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2\" (UID: \"18798180-d3f0-4a87-8226-1b66d45a76bc\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"
Nov 02 12:19:40.534598 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.534576    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/18798180-d3f0-4a87-8226-1b66d45a76bc-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2\" (UID: \"18798180-d3f0-4a87-8226-1b66d45a76bc\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"
Nov 02 12:19:40.592944 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.592877    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwtnc\" (UniqueName: \"kubernetes.io/projected/18798180-d3f0-4a87-8226-1b66d45a76bc-kube-api-access-mwtnc\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2\" (UID: \"18798180-d3f0-4a87-8226-1b66d45a76bc\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"
Nov 02 12:19:40.649801 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:40.649771    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"
Nov 02 12:19:41.026019 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:41.025992    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"]
Nov 02 12:19:41.026536 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:19:41.026507    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18798180_d3f0_4a87_8226_1b66d45a76bc.slice/crio-04f285c1ad55b82b42a648502e8ad3b34607d0ae2a754948e500124cb72c1b49 WatchSource:0}: Error finding container 04f285c1ad55b82b42a648502e8ad3b34607d0ae2a754948e500124cb72c1b49: Status 404 returned error can't find the container with id 04f285c1ad55b82b42a648502e8ad3b34607d0ae2a754948e500124cb72c1b49
Nov 02 12:19:41.807583 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:41.807548    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2" event={"ID":"18798180-d3f0-4a87-8226-1b66d45a76bc","Type":"ContainerStarted","Data":"8fc898b8c6b0e3b90309d3c2bf917b973eaa7f6af2f3df2e27c0a9803d070ca3"}
Nov 02 12:19:41.807947 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:41.807608    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2" event={"ID":"18798180-d3f0-4a87-8226-1b66d45a76bc","Type":"ContainerStarted","Data":"04f285c1ad55b82b42a648502e8ad3b34607d0ae2a754948e500124cb72c1b49"}
Nov 02 12:19:41.807947 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:41.807660    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"
Nov 02 12:19:41.930493 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:41.930440    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2" podStartSLOduration=1.930427297 podStartE2EDuration="1.930427297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:19:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:19:41.927639281 +0000 UTC m=+1408.498989035" watchObservedRunningTime="2026-11-02 12:19:41.930427297 +0000 UTC m=+1408.501777054"
Nov 02 12:19:43.083685 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.083645    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"]
Nov 02 12:19:43.087990 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.087965    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.178322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.176225    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6a432197-openshift-default-dockercfg-n865n\""
Nov 02 12:19:43.178322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.176992    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-istio-envoy\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.178322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.177066    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6929f886-c638-4507-989e-74f2750c9bb6-istiod-ca-cert\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.178322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.177116    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6929f886-c638-4507-989e-74f2750c9bb6-istio-podinfo\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.178322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.177203    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6929f886-c638-4507-989e-74f2750c9bb6-istio-token\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.178322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.177229    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-credential-socket\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.178322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.177276    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-istio-data\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.178322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.177343    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-workload-socket\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.178322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.177374    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd2hr\" (UniqueName: \"kubernetes.io/projected/6929f886-c638-4507-989e-74f2750c9bb6-kube-api-access-wd2hr\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.178322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.177425    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-workload-certs\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.230245 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.230216    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"]
Nov 02 12:19:43.278329 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.278257    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6929f886-c638-4507-989e-74f2750c9bb6-istio-podinfo\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.278514 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.278359    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6929f886-c638-4507-989e-74f2750c9bb6-istio-token\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.278514 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.278383    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-credential-socket\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.278514 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.278417    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-istio-data\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.278514 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.278473    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-workload-socket\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.278514 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.278510    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd2hr\" (UniqueName: \"kubernetes.io/projected/6929f886-c638-4507-989e-74f2750c9bb6-kube-api-access-wd2hr\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.278788 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.278564    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-workload-certs\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.278788 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.278661    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-istio-envoy\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.278788 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.278725    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6929f886-c638-4507-989e-74f2750c9bb6-istiod-ca-cert\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.278946 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.278840    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-istio-data\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.279093 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.279064    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-workload-socket\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.279151 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.279074    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-credential-socket\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.279351 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.279334    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-workload-certs\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.279453 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.279434    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6929f886-c638-4507-989e-74f2750c9bb6-istiod-ca-cert\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.281016 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.280992    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-istio-envoy\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.281399 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.281380    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6929f886-c638-4507-989e-74f2750c9bb6-istio-podinfo\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.349899    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6929f886-c638-4507-989e-74f2750c9bb6-istio-token\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.367377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.361998    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd2hr\" (UniqueName: \"kubernetes.io/projected/6929f886-c638-4507-989e-74f2750c9bb6-kube-api-access-wd2hr\") pod \"e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") " pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.406860 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.406547    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:43.846917 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:19:43.846883    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6929f886_c638_4507_989e_74f2750c9bb6.slice/crio-8ebace5e174e07c674a3870b25dd5a4fff28f7180f6b72d982bf176a2f6cb259 WatchSource:0}: Error finding container 8ebace5e174e07c674a3870b25dd5a4fff28f7180f6b72d982bf176a2f6cb259: Status 404 returned error can't find the container with id 8ebace5e174e07c674a3870b25dd5a4fff28f7180f6b72d982bf176a2f6cb259
Nov 02 12:19:43.849076 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.849043    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"]
Nov 02 12:19:43.849356 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.849292    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:19:43.849408 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.849389    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:19:43.849452 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:43.849420    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:19:44.263395 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.263360    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"]
Nov 02 12:19:44.268416 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.268387    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"
Nov 02 12:19:44.324333 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.322646    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-6ac03f\"/\"kube-root-ca.crt\""
Nov 02 12:19:44.333316 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.329514    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-6ac03f\"/\"openshift-service-ca.crt\""
Nov 02 12:19:44.333316 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.330334    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-6ac03f\"/\"payload-processing-external-model-e2e--75a83f02-dockercfg-2bmrj\""
Nov 02 12:19:44.333316 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.330433    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-6ac03f\"/\"agc-cred-c94dcc9533fe\""
Nov 02 12:19:44.333316 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.330442    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-6ac03f\"/\"routing-overlay\""
Nov 02 12:19:44.333316 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.330537    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-6ac03f\"/\"payload-processing-external-model-plugins-e2e-extmod-a-6ac03f\""
Nov 02 12:19:44.333316 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.332976    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"]
Nov 02 12:19:44.418322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.418099    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e118bf7e-8161-456b-8502-911eb43e6887-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25\" (UID: \"e118bf7e-8161-456b-8502-911eb43e6887\") " pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"
Nov 02 12:19:44.418322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.418238    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e118bf7e-8161-456b-8502-911eb43e6887-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25\" (UID: \"e118bf7e-8161-456b-8502-911eb43e6887\") " pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"
Nov 02 12:19:44.432351 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.418290    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25\" (UID: \"e118bf7e-8161-456b-8502-911eb43e6887\") " pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"
Nov 02 12:19:44.432351 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.419354    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpn5d\" (UniqueName: \"kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-kube-api-access-rpn5d\") pod \"payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25\" (UID: \"e118bf7e-8161-456b-8502-911eb43e6887\") " pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"
Nov 02 12:19:44.555377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.554652    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpn5d\" (UniqueName: \"kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-kube-api-access-rpn5d\") pod \"payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25\" (UID: \"e118bf7e-8161-456b-8502-911eb43e6887\") " pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"
Nov 02 12:19:44.555377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.554738    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e118bf7e-8161-456b-8502-911eb43e6887-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25\" (UID: \"e118bf7e-8161-456b-8502-911eb43e6887\") " pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"
Nov 02 12:19:44.555377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.554857    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e118bf7e-8161-456b-8502-911eb43e6887-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25\" (UID: \"e118bf7e-8161-456b-8502-911eb43e6887\") " pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"
Nov 02 12:19:44.555377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.554907    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25\" (UID: \"e118bf7e-8161-456b-8502-911eb43e6887\") " pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"
Nov 02 12:19:44.575935 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.570682    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e118bf7e-8161-456b-8502-911eb43e6887-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25\" (UID: \"e118bf7e-8161-456b-8502-911eb43e6887\") " pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"
Nov 02 12:19:44.600321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.599958    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e118bf7e-8161-456b-8502-911eb43e6887-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25\" (UID: \"e118bf7e-8161-456b-8502-911eb43e6887\") " pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"
Nov 02 12:19:44.613319 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.611197    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25\" (UID: \"e118bf7e-8161-456b-8502-911eb43e6887\") " pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"
Nov 02 12:19:44.638323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.637003    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpn5d\" (UniqueName: \"kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-kube-api-access-rpn5d\") pod \"payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25\" (UID: \"e118bf7e-8161-456b-8502-911eb43e6887\") " pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"
Nov 02 12:19:44.846529 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.845483    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd" event={"ID":"6929f886-c638-4507-989e-74f2750c9bb6","Type":"ContainerStarted","Data":"515f8297836cc66d087e308006d234fcde215bc09079dc524709e613644da8c6"}
Nov 02 12:19:44.846529 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.845548    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd" event={"ID":"6929f886-c638-4507-989e-74f2750c9bb6","Type":"ContainerStarted","Data":"8ebace5e174e07c674a3870b25dd5a4fff28f7180f6b72d982bf176a2f6cb259"}
Nov 02 12:19:44.889949 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:44.889918    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"
Nov 02 12:19:45.042127 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:45.018594    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd" podStartSLOduration=2.018572152 podStartE2EDuration="2.018572152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:19:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:19:45.004388098 +0000 UTC m=+1411.575737869" watchObservedRunningTime="2026-11-02 12:19:45.018572152 +0000 UTC m=+1411.589921912"
Nov 02 12:19:45.386203 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:45.386156    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"]
Nov 02 12:19:45.413403 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:45.413346    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:45.430662 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:19:45.422681    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode118bf7e_8161_456b_8502_911eb43e6887.slice/crio-759c0edbd19fa7fa422cb008ac17cfe5f574ee7508f3af792f3448cf2a6d179a WatchSource:0}: Error finding container 759c0edbd19fa7fa422cb008ac17cfe5f574ee7508f3af792f3448cf2a6d179a: Status 404 returned error can't find the container with id 759c0edbd19fa7fa422cb008ac17cfe5f574ee7508f3af792f3448cf2a6d179a
Nov 02 12:19:45.446922 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:45.445200    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:45.898378 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:45.890187    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25" event={"ID":"e118bf7e-8161-456b-8502-911eb43e6887","Type":"ContainerStarted","Data":"b62ff879fa7b4f9befa49b125abfd297e0a171c6c6128452c53d4e8a4553ee49"}
Nov 02 12:19:45.898378 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:45.890248    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"
Nov 02 12:19:45.898378 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:45.890271    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25" event={"ID":"e118bf7e-8161-456b-8502-911eb43e6887","Type":"ContainerStarted","Data":"759c0edbd19fa7fa422cb008ac17cfe5f574ee7508f3af792f3448cf2a6d179a"}
Nov 02 12:19:45.898378 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:45.890620    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:45.899012 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:45.898988    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:19:45.946425 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:19:45.946344    4942 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-6ac03f/agc-cred-c94dcc9533fe: secret "agc-cred-c94dcc9533fe" not found
Nov 02 12:19:45.948158 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:19:45.948139    4942 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25: secret "agc-cred-c94dcc9533fe" not found
Nov 02 12:19:45.948390 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:19:45.948376    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-provider-credentials podName:e118bf7e-8161-456b-8502-911eb43e6887 nodeName:}" failed. No retries permitted until 2026-11-02 12:19:46.448353473 +0000 UTC m=+1413.019703223 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25" (UID: "e118bf7e-8161-456b-8502-911eb43e6887") : secret "agc-cred-c94dcc9533fe" not found
Nov 02 12:19:46.168453 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:46.168386    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25" podStartSLOduration=2.168361501 podStartE2EDuration="2.168361501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:19:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:19:46.019559957 +0000 UTC m=+1412.590909715" watchObservedRunningTime="2026-11-02 12:19:46.168361501 +0000 UTC m=+1412.739711259"
Nov 02 12:19:46.518319 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:19:46.518097    4942 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-6ac03f/agc-cred-c94dcc9533fe: secret "agc-cred-c94dcc9533fe" not found
Nov 02 12:19:46.518319 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:19:46.518128    4942 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25: secret "agc-cred-c94dcc9533fe" not found
Nov 02 12:19:46.518319 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:19:46.518208    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-provider-credentials podName:e118bf7e-8161-456b-8502-911eb43e6887 nodeName:}" failed. No retries permitted until 2026-11-02 12:19:47.518186587 +0000 UTC m=+1414.089536342 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25" (UID: "e118bf7e-8161-456b-8502-911eb43e6887") : secret "agc-cred-c94dcc9533fe" not found
Nov 02 12:19:47.349432 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:47.349383    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb" podUID="ca691fd2-312b-49cd-9bdd-4010c8fb10fc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.87:8000/ready\": context deadline exceeded"
Nov 02 12:19:47.392324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:47.390624    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f" podUID="5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:19:47.621938 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:47.621888    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28" podUID="f3d8189b-983b-488a-9419-788fdf6f93cb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:19:47.659324 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:19:47.642151    4942 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-6ac03f/agc-cred-c94dcc9533fe: secret "agc-cred-c94dcc9533fe" not found
Nov 02 12:19:47.659324 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:19:47.642182    4942 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25: secret "agc-cred-c94dcc9533fe" not found
Nov 02 12:19:47.659324 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:19:47.642251    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-provider-credentials podName:e118bf7e-8161-456b-8502-911eb43e6887 nodeName:}" failed. No retries permitted until 2026-11-02 12:19:49.642230804 +0000 UTC m=+1416.213580555 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25" (UID: "e118bf7e-8161-456b-8502-911eb43e6887") : secret "agc-cred-c94dcc9533fe" not found
Nov 02 12:19:47.867874 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:47.850978    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"]
Nov 02 12:19:48.410003 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:48.408501    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f" podUID="5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:19:48.410003 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:48.409375    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:19:48.655324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:48.654462    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28" podUID="f3d8189b-983b-488a-9419-788fdf6f93cb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:19:48.675370 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:48.665601    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:19:49.154320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:49.153627    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw" podUID="242aa9d0-49d5-4981-ad74-0f7b95635e6a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:19:49.273962 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:49.273883    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:19:49.275741 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:49.275705    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:19:49.663633 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:19:49.663594    4942 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-6ac03f/agc-cred-c94dcc9533fe: secret "agc-cred-c94dcc9533fe" not found
Nov 02 12:19:49.663633 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:19:49.663627    4942 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25: secret "agc-cred-c94dcc9533fe" not found
Nov 02 12:19:49.664162 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:19:49.663693    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-provider-credentials podName:e118bf7e-8161-456b-8502-911eb43e6887 nodeName:}" failed. No retries permitted until 2026-11-02 12:19:53.663673471 +0000 UTC m=+1420.235023225 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25" (UID: "e118bf7e-8161-456b-8502-911eb43e6887") : secret "agc-cred-c94dcc9533fe" not found
Nov 02 12:19:52.816523 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:52.816490    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"
Nov 02 12:19:53.085077 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:53.084988    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25" podUID="e118bf7e-8161-456b-8502-911eb43e6887" containerName="payload-processing" containerID="cri-o://b62ff879fa7b4f9befa49b125abfd297e0a171c6c6128452c53d4e8a4553ee49" gracePeriod=25
Nov 02 12:19:53.222483 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:53.222448    4942 generic.go:356] "Generic (PLEG): container finished" podID="e118bf7e-8161-456b-8502-911eb43e6887" containerID="b62ff879fa7b4f9befa49b125abfd297e0a171c6c6128452c53d4e8a4553ee49" exitCode=0
Nov 02 12:19:53.222630 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:53.222516    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25" event={"ID":"e118bf7e-8161-456b-8502-911eb43e6887","Type":"ContainerDied","Data":"b62ff879fa7b4f9befa49b125abfd297e0a171c6c6128452c53d4e8a4553ee49"}
Nov 02 12:19:53.355854 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:53.355825    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"
Nov 02 12:19:53.508558 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:53.508520    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e118bf7e-8161-456b-8502-911eb43e6887-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e118bf7e-8161-456b-8502-911eb43e6887-plugins-config-volume\") pod \"e118bf7e-8161-456b-8502-911eb43e6887\" (UID: \"e118bf7e-8161-456b-8502-911eb43e6887\") "
Nov 02 12:19:53.508742 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:53.508568    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-provider-credentials\" (UniqueName: \"kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-provider-credentials\") pod \"e118bf7e-8161-456b-8502-911eb43e6887\" (UID: \"e118bf7e-8161-456b-8502-911eb43e6887\") "
Nov 02 12:19:53.508742 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:53.508691    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e118bf7e-8161-456b-8502-911eb43e6887-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e118bf7e-8161-456b-8502-911eb43e6887-routing-overlay-volume\") pod \"e118bf7e-8161-456b-8502-911eb43e6887\" (UID: \"e118bf7e-8161-456b-8502-911eb43e6887\") "
Nov 02 12:19:53.508742 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:53.508733    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-kube-api-access-rpn5d\" (UniqueName: \"kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-kube-api-access-rpn5d\") pod \"e118bf7e-8161-456b-8502-911eb43e6887\" (UID: \"e118bf7e-8161-456b-8502-911eb43e6887\") "
Nov 02 12:19:53.509036 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:53.509004    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e118bf7e-8161-456b-8502-911eb43e6887-plugins-config-volume" pod "e118bf7e-8161-456b-8502-911eb43e6887" (UID: "e118bf7e-8161-456b-8502-911eb43e6887"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:19:53.509215 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:53.509062    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e118bf7e-8161-456b-8502-911eb43e6887-routing-overlay-volume" pod "e118bf7e-8161-456b-8502-911eb43e6887" (UID: "e118bf7e-8161-456b-8502-911eb43e6887"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:19:53.510984 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:53.510956    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-kube-api-access-rpn5d" pod "e118bf7e-8161-456b-8502-911eb43e6887" (UID: "e118bf7e-8161-456b-8502-911eb43e6887"). InnerVolumeSpecName "kube-api-access-rpn5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:19:53.511265 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:53.511238    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-provider-credentials" pod "e118bf7e-8161-456b-8502-911eb43e6887" (UID: "e118bf7e-8161-456b-8502-911eb43e6887"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:19:53.610280 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:53.610254    4942 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-provider-credentials\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:53.610409 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:53.610281    4942 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e118bf7e-8161-456b-8502-911eb43e6887-routing-overlay-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:53.610409 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:53.610293    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpn5d\" (UniqueName: \"kubernetes.io/projected/e118bf7e-8161-456b-8502-911eb43e6887-kube-api-access-rpn5d\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:53.610409 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:53.610329    4942 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e118bf7e-8161-456b-8502-911eb43e6887-plugins-config-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:19:54.229468 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:54.229438    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"
Nov 02 12:19:54.229915 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:54.229486    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25" event={"ID":"e118bf7e-8161-456b-8502-911eb43e6887","Type":"ContainerDied","Data":"759c0edbd19fa7fa422cb008ac17cfe5f574ee7508f3af792f3448cf2a6d179a"}
Nov 02 12:19:54.229915 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:54.229550    4942 scope.go:122] "RemoveContainer" containerID="b62ff879fa7b4f9befa49b125abfd297e0a171c6c6128452c53d4e8a4553ee49"
Nov 02 12:19:54.393707 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:54.393676    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"]
Nov 02 12:19:54.442704 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:54.442675    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-6ac03f/payload-processing-external-model-e2e-extmod-a-6ac03f-56f646k25"]
Nov 02 12:19:55.548549 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:19:55.548506    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e118bf7e-8161-456b-8502-911eb43e6887" path="/var/lib/kubelet/pods/e118bf7e-8161-456b-8502-911eb43e6887/volumes"
Nov 02 12:20:00.983507 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:20:00.983426    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:20:04.129514 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:04.129446    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw" podUID="242aa9d0-49d5-4981-ad74-0f7b95635e6a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:20:04.279359 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:04.275342    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28" podUID="f3d8189b-983b-488a-9419-788fdf6f93cb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:20:04.481532 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:04.481380    4942 prober.go:122] "Probe failed" probeType="Liveness" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28" podUID="f3d8189b-983b-488a-9419-788fdf6f93cb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/health\": context deadline exceeded"
Nov 02 12:20:12.055491 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:12.055453    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw"]
Nov 02 12:20:12.104851 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:12.104816    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-68a5a075-nvfzw"]
Nov 02 12:20:13.543148 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:13.543111    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="62c1a154-3a84-4f8b-bbc1-f4c28740ee31" path="/var/lib/kubelet/pods/62c1a154-3a84-4f8b-bbc1-f4c28740ee31/volumes"
Nov 02 12:20:19.397040 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:19.397001    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"]
Nov 02 12:20:19.397558 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:19.397250    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62" podUID="37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" containerName="istio-proxy" containerID="cri-o://e48ba6c6d932d48fa85f15a7b0364f69a2296dffd5e2e7171c146bcdf62cf321" gracePeriod=30
Nov 02 12:20:21.209283 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:21.209243    4942 patch_prober.go:29] interesting pod/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 12:20:21.209678 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:21.209346    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62" podUID="37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 12:20:24.546942 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.546912    4942 generic.go:356] "Generic (PLEG): container finished" podID="37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" containerID="e48ba6c6d932d48fa85f15a7b0364f69a2296dffd5e2e7171c146bcdf62cf321" exitCode=0
Nov 02 12:20:24.547351 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.546983    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62" event={"ID":"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3","Type":"ContainerDied","Data":"e48ba6c6d932d48fa85f15a7b0364f69a2296dffd5e2e7171c146bcdf62cf321"}
Nov 02 12:20:24.665863 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.665805    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:20:24.754506 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.754475    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-workload-certs\") pod \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") "
Nov 02 12:20:24.754685 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.754531    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-workload-socket\") pod \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") "
Nov 02 12:20:24.754685 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.754587    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-podinfo\") pod \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") "
Nov 02 12:20:24.754685 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.754627    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istiod-ca-cert\") pod \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") "
Nov 02 12:20:24.754852 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.754726    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-credential-socket\") pod \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") "
Nov 02 12:20:24.754852 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.754763    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-data\") pod \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") "
Nov 02 12:20:24.754852 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.754769    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-workload-certs" pod "37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" (UID: "37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:20:24.754852 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.754799    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-kube-api-access-nw42d\" (UniqueName: \"kubernetes.io/projected/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-kube-api-access-nw42d\") pod \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") "
Nov 02 12:20:24.755060 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.754857    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-workload-socket" pod "37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" (UID: "37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:20:24.755060 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.754911    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-token\" (UniqueName: \"kubernetes.io/projected/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-token\") pod \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") "
Nov 02 12:20:24.755060 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.754950    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-envoy\") pod \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\" (UID: \"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3\") "
Nov 02 12:20:24.755060 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.755014    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-credential-socket" pod "37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" (UID: "37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:20:24.755060 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.755012    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istiod-ca-cert" pod "37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" (UID: "37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:20:24.755344 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.755068    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-data" pod "37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" (UID: "37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:20:24.755552 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.755514    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:20:24.755552 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.755541    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:20:24.755639 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.755554    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:20:24.755639 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.755568    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:20:24.755639 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.755582    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:20:24.757006 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.756984    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-token" pod "37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" (UID: "37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:20:24.757094 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.757049    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-envoy" pod "37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" (UID: "37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:20:24.757376 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.757358    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-kube-api-access-nw42d" pod "37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" (UID: "37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3"). InnerVolumeSpecName "kube-api-access-nw42d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:20:24.757451 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.757396    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-podinfo" pod "37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" (UID: "37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:20:24.856765 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.856733    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nw42d\" (UniqueName: \"kubernetes.io/projected/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-kube-api-access-nw42d\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:20:24.856765 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.856765    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:20:24.856989 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.856779    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:20:24.856989 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:24.856792    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:20:25.554014 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.553991    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"
Nov 02 12:20:25.554418 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.554027    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62" event={"ID":"37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3","Type":"ContainerDied","Data":"b7271e450c31cff0a5a6fa24bdd25ed86b2f779eeb1f82adf667b7b48e4575cf"}
Nov 02 12:20:25.554418 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.554089    4942 scope.go:122] "RemoveContainer" containerID="e48ba6c6d932d48fa85f15a7b0364f69a2296dffd5e2e7171c146bcdf62cf321"
Nov 02 12:20:25.743290 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.743260    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"]
Nov 02 12:20:25.743798 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.743784    4942 cpu_manager.go:414] "containerMap: removing container" podUID="e118bf7e-8161-456b-8502-911eb43e6887" containerName="payload-processing"
Nov 02 12:20:25.743844 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.743799    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e118bf7e-8161-456b-8502-911eb43e6887" containerName="payload-processing"
Nov 02 12:20:25.743844 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.743824    4942 cpu_manager.go:414] "containerMap: removing container" podUID="37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" containerName="istio-proxy"
Nov 02 12:20:25.743844 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.743829    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" containerName="istio-proxy"
Nov 02 12:20:25.743935 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.743911    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" containerName="istio-proxy"
Nov 02 12:20:25.743935 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.743925    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="e118bf7e-8161-456b-8502-911eb43e6887" containerName="payload-processing"
Nov 02 12:20:25.748684 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.748662    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"
Nov 02 12:20:25.770965 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.770941    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"]
Nov 02 12:20:25.794547 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.794517    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-036f88-openshift-default-dbc8cc695-knp62"]
Nov 02 12:20:25.802019 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.801944    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6a432197-serving-cert\""
Nov 02 12:20:25.806368 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.806345    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"]
Nov 02 12:20:25.868311 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.868272    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-metrics-certs\") pod \"maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw\" (UID: \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"
Nov 02 12:20:25.868450 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.868402    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-maas-api-tls\") pod \"maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw\" (UID: \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"
Nov 02 12:20:25.868566 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.868548    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6pdw\" (UniqueName: \"kubernetes.io/projected/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-kube-api-access-r6pdw\") pod \"maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw\" (UID: \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"
Nov 02 12:20:25.868615 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.868586    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-openshift-service-ca\") pod \"maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw\" (UID: \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"
Nov 02 12:20:25.969702 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.969664    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6pdw\" (UniqueName: \"kubernetes.io/projected/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-kube-api-access-r6pdw\") pod \"maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw\" (UID: \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"
Nov 02 12:20:25.969702 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.969706    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-openshift-service-ca\") pod \"maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw\" (UID: \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"
Nov 02 12:20:25.969970 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.969824    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-metrics-certs\") pod \"maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw\" (UID: \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"
Nov 02 12:20:25.969970 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.969873    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-maas-api-tls\") pod \"maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw\" (UID: \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"
Nov 02 12:20:25.970633 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.970603    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-openshift-service-ca\") pod \"maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw\" (UID: \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"
Nov 02 12:20:25.973160 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.973134    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-maas-api-tls\") pod \"maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw\" (UID: \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"
Nov 02 12:20:25.973272 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:25.973134    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-metrics-certs\") pod \"maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw\" (UID: \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"
Nov 02 12:20:26.030826 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:26.030799    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6pdw\" (UniqueName: \"kubernetes.io/projected/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-kube-api-access-r6pdw\") pod \"maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw\" (UID: \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"
Nov 02 12:20:26.064008 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:26.063947    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"
Nov 02 12:20:26.445363 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:26.445335    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"]
Nov 02 12:20:26.446280 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:20:26.446251    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e2c7ff8_8b9a_4bcf_9b7c_418ecf231b4a.slice/crio-199e7675c970228e0d8ce8ecdf660057bb99867436d121a0200e679fde653c34 WatchSource:0}: Error finding container 199e7675c970228e0d8ce8ecdf660057bb99867436d121a0200e679fde653c34: Status 404 returned error can't find the container with id 199e7675c970228e0d8ce8ecdf660057bb99867436d121a0200e679fde653c34
Nov 02 12:20:26.447677 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:26.447656    4942 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 12:20:26.563995 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:26.563956    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw" event={"ID":"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a","Type":"ContainerStarted","Data":"199e7675c970228e0d8ce8ecdf660057bb99867436d121a0200e679fde653c34"}
Nov 02 12:20:27.543951 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:27.543920    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3" path="/var/lib/kubelet/pods/37f6c8a7-d9a5-4d1d-b46a-8ec120b1cea3/volumes"
Nov 02 12:20:27.573047 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:27.573012    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw" event={"ID":"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a","Type":"ContainerStarted","Data":"43abfe8cb138fd9445cccace1ca1798b9aa77e139cc4475997e235bbcad080e4"}
Nov 02 12:20:27.573513 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:27.573089    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"
Nov 02 12:20:27.730377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:27.725283    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw" podStartSLOduration=1.73608333 podStartE2EDuration="2.725261338s" totalImagesPullingTime="989.178008ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:20:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:20:27.722339089 +0000 UTC m=+1454.293688846" watchObservedRunningTime="2026-11-02 12:20:27.725261338 +0000 UTC m=+1454.296611097"
Nov 02 12:20:30.594706 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:30.594364    4942 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f" podUID="5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:20:31.068347 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:20:31.067924    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:20:33.592545 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:33.592515    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"
Nov 02 12:20:56.967656 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:56.967626    4942 generic.go:356] "Generic (PLEG): container finished" podID="6929f886-c638-4507-989e-74f2750c9bb6" containerID="515f8297836cc66d087e308006d234fcde215bc09079dc524709e613644da8c6" exitCode=0
Nov 02 12:20:56.968113 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:56.967702    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd" event={"ID":"6929f886-c638-4507-989e-74f2750c9bb6","Type":"ContainerDied","Data":"515f8297836cc66d087e308006d234fcde215bc09079dc524709e613644da8c6"}
Nov 02 12:20:56.968186 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:56.968119    4942 scope.go:122] "RemoveContainer" containerID="515f8297836cc66d087e308006d234fcde215bc09079dc524709e613644da8c6"
Nov 02 12:20:56.969893 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:56.969855    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:20:56.969968 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:56.969951    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:20:56.970016 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:56.970001    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:20:57.989150 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:57.989114    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd" event={"ID":"6929f886-c638-4507-989e-74f2750c9bb6","Type":"ContainerStarted","Data":"f0414cfe9d263bfaacd530f704efd5b0095e1487b440b6b0535b90ba91abd520"}
Nov 02 12:20:58.407537 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:58.407501    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:20:58.409124 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:58.409093    4942 patch_prober.go:29] interesting pod/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 02 12:20:58.409282 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:58.409149    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd" podUID="6929f886-c638-4507-989e-74f2750c9bb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 02 12:20:59.408689 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:59.408653    4942 patch_prober.go:29] interesting pod/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 02 12:20:59.409645 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:20:59.409265    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd" podUID="6929f886-c638-4507-989e-74f2750c9bb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 02 12:21:00.417320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:00.416741    4942 patch_prober.go:29] interesting pod/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 02 12:21:00.417320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:00.416808    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd" podUID="6929f886-c638-4507-989e-74f2750c9bb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 02 12:21:00.891832 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:00.891800    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:21:01.113804 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:21:01.113710    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:21:01.282337 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:01.281472    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28" podUID="f3d8189b-983b-488a-9419-788fdf6f93cb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": context deadline exceeded"
Nov 02 12:21:01.288436 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:01.288383    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f" podUID="5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Nov 02 12:21:01.292729 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:01.292687    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb" podUID="ca691fd2-312b-49cd-9bdd-4010c8fb10fc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.87:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:21:01.419381 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:01.407442    4942 patch_prober.go:29] interesting pod/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 02 12:21:01.419381 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:01.407505    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd" podUID="6929f886-c638-4507-989e-74f2750c9bb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 02 12:21:02.408156 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:02.408119    4942 patch_prober.go:29] interesting pod/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 02 12:21:02.408350 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:02.408183    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd" podUID="6929f886-c638-4507-989e-74f2750c9bb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 02 12:21:03.407834 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:03.407788    4942 patch_prober.go:29] interesting pod/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 02 12:21:03.408357 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:03.407857    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd" podUID="6929f886-c638-4507-989e-74f2750c9bb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 02 12:21:04.412091 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:04.412058    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:21:04.413124 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:04.413103    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:21:13.747569 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:13.747541    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:21:13.756327 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:13.756277    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:21:13.907009 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:13.906975    4942 scope.go:122] "RemoveContainer" containerID="3a271b864097048db44f80afb55300c3dec54e6c8880585b57ad8d24ef3610ee"
Nov 02 12:21:16.459158 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:16.455745    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"]
Nov 02 12:21:16.469013 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:16.468284    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:21:16.538741 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:16.538675    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"]
Nov 02 12:21:16.560324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:16.543691    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a956fba-d8ca-4732-a017-dcbcddca6aa9-maas-api-tls\") pod \"maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z\" (UID: \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:21:16.560324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:16.543806    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a956fba-d8ca-4732-a017-dcbcddca6aa9-openshift-service-ca\") pod \"maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z\" (UID: \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:21:16.560324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:16.543901    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a956fba-d8ca-4732-a017-dcbcddca6aa9-metrics-certs\") pod \"maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z\" (UID: \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:21:16.560324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:16.543971    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mh7d\" (UniqueName: \"kubernetes.io/projected/3a956fba-d8ca-4732-a017-dcbcddca6aa9-kube-api-access-4mh7d\") pod \"maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z\" (UID: \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:21:16.560939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:16.560909    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-fbf1444a-serving-cert\""
Nov 02 12:21:16.831373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:16.824191    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a956fba-d8ca-4732-a017-dcbcddca6aa9-metrics-certs\") pod \"maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z\" (UID: \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:21:16.831373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:16.824266    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mh7d\" (UniqueName: \"kubernetes.io/projected/3a956fba-d8ca-4732-a017-dcbcddca6aa9-kube-api-access-4mh7d\") pod \"maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z\" (UID: \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:21:16.831373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:16.824381    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a956fba-d8ca-4732-a017-dcbcddca6aa9-maas-api-tls\") pod \"maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z\" (UID: \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:21:16.831373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:16.824469    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a956fba-d8ca-4732-a017-dcbcddca6aa9-openshift-service-ca\") pod \"maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z\" (UID: \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:21:16.831373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:16.825366    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a956fba-d8ca-4732-a017-dcbcddca6aa9-openshift-service-ca\") pod \"maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z\" (UID: \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:21:16.831373 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:21:16.829445    4942 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-serving-cert: secret "maas-api-e2e-ait-fbf1444a-serving-cert" not found
Nov 02 12:21:16.831373 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:21:16.829509    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a956fba-d8ca-4732-a017-dcbcddca6aa9-maas-api-tls podName:3a956fba-d8ca-4732-a017-dcbcddca6aa9 nodeName:}" failed. No retries permitted until 2026-11-02 12:21:17.329488892 +0000 UTC m=+1503.900838645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3a956fba-d8ca-4732-a017-dcbcddca6aa9-maas-api-tls") pod "maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z" (UID: "3a956fba-d8ca-4732-a017-dcbcddca6aa9") : secret "maas-api-e2e-ait-fbf1444a-serving-cert" not found
Nov 02 12:21:16.954323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:16.951092    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mh7d\" (UniqueName: \"kubernetes.io/projected/3a956fba-d8ca-4732-a017-dcbcddca6aa9-kube-api-access-4mh7d\") pod \"maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z\" (UID: \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:21:16.970323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:16.968577    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a956fba-d8ca-4732-a017-dcbcddca6aa9-metrics-certs\") pod \"maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z\" (UID: \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:21:17.344464 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:17.344427    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a956fba-d8ca-4732-a017-dcbcddca6aa9-maas-api-tls\") pod \"maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z\" (UID: \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:21:17.346837 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:17.346812    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a956fba-d8ca-4732-a017-dcbcddca6aa9-maas-api-tls\") pod \"maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z\" (UID: \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:21:17.398913 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:17.398885    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:21:17.775075 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:17.775044    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"]
Nov 02 12:21:17.775952 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:21:17.775924    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a956fba_d8ca_4732_a017_dcbcddca6aa9.slice/crio-c18cd71d8235b650047a8d8b3ab5b2040cfcd61125dbf0833253415d7ddf7bb9 WatchSource:0}: Error finding container c18cd71d8235b650047a8d8b3ab5b2040cfcd61125dbf0833253415d7ddf7bb9: Status 404 returned error can't find the container with id c18cd71d8235b650047a8d8b3ab5b2040cfcd61125dbf0833253415d7ddf7bb9
Nov 02 12:21:18.222755 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:18.222720    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z" event={"ID":"3a956fba-d8ca-4732-a017-dcbcddca6aa9","Type":"ContainerStarted","Data":"c18cd71d8235b650047a8d8b3ab5b2040cfcd61125dbf0833253415d7ddf7bb9"}
Nov 02 12:21:19.234712 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:19.234672    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z" event={"ID":"3a956fba-d8ca-4732-a017-dcbcddca6aa9","Type":"ContainerStarted","Data":"1ce510f79b8f36beda00cff03a18a3292e5ee41c20eaf3fbe98af9731d83a7ab"}
Nov 02 12:21:19.235199 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:19.234818    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:21:19.361172 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:19.361106    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z" podStartSLOduration=2.526198071 podStartE2EDuration="3.361086554s" totalImagesPullingTime="834.888483ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:21:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:21:19.358738243 +0000 UTC m=+1505.930088003" watchObservedRunningTime="2026-11-02 12:21:19.361086554 +0000 UTC m=+1505.932436312"
Nov 02 12:21:25.245972 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:21:25.245940    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:21:31.171694 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:21:31.171653    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:22:00.983586 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:22:00.983544    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:22:30.983764 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:22:30.983690    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:22:46.495925 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:46.495887    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"]
Nov 02 12:22:46.496495 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:46.496095    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z" podUID="3a956fba-d8ca-4732-a017-dcbcddca6aa9" containerName="maas-api" containerID="cri-o://1ce510f79b8f36beda00cff03a18a3292e5ee41c20eaf3fbe98af9731d83a7ab" gracePeriod=30
Nov 02 12:22:46.779645 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:46.779613    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:22:46.828636 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:46.828605    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a956fba-d8ca-4732-a017-dcbcddca6aa9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a956fba-d8ca-4732-a017-dcbcddca6aa9-openshift-service-ca\") pod \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\" (UID: \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\") "
Nov 02 12:22:46.828769 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:46.828710    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3a956fba-d8ca-4732-a017-dcbcddca6aa9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a956fba-d8ca-4732-a017-dcbcddca6aa9-metrics-certs\") pod \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\" (UID: \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\") "
Nov 02 12:22:46.828769 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:46.828749    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a956fba-d8ca-4732-a017-dcbcddca6aa9-kube-api-access-4mh7d\" (UniqueName: \"kubernetes.io/projected/3a956fba-d8ca-4732-a017-dcbcddca6aa9-kube-api-access-4mh7d\") pod \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\" (UID: \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\") "
Nov 02 12:22:46.828845 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:46.828829    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3a956fba-d8ca-4732-a017-dcbcddca6aa9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a956fba-d8ca-4732-a017-dcbcddca6aa9-maas-api-tls\") pod \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\" (UID: \"3a956fba-d8ca-4732-a017-dcbcddca6aa9\") "
Nov 02 12:22:46.828990 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:46.828955    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a956fba-d8ca-4732-a017-dcbcddca6aa9-openshift-service-ca" pod "3a956fba-d8ca-4732-a017-dcbcddca6aa9" (UID: "3a956fba-d8ca-4732-a017-dcbcddca6aa9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:22:46.829323 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:46.829282    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a956fba-d8ca-4732-a017-dcbcddca6aa9-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:22:46.830840 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:46.830816    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a956fba-d8ca-4732-a017-dcbcddca6aa9-metrics-certs" pod "3a956fba-d8ca-4732-a017-dcbcddca6aa9" (UID: "3a956fba-d8ca-4732-a017-dcbcddca6aa9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:22:46.830918 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:46.830872    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a956fba-d8ca-4732-a017-dcbcddca6aa9-kube-api-access-4mh7d" pod "3a956fba-d8ca-4732-a017-dcbcddca6aa9" (UID: "3a956fba-d8ca-4732-a017-dcbcddca6aa9"). InnerVolumeSpecName "kube-api-access-4mh7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:22:46.831046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:46.831023    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a956fba-d8ca-4732-a017-dcbcddca6aa9-maas-api-tls" pod "3a956fba-d8ca-4732-a017-dcbcddca6aa9" (UID: "3a956fba-d8ca-4732-a017-dcbcddca6aa9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:22:46.930922 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:46.930885    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a956fba-d8ca-4732-a017-dcbcddca6aa9-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:22:46.930922 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:46.930924    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mh7d\" (UniqueName: \"kubernetes.io/projected/3a956fba-d8ca-4732-a017-dcbcddca6aa9-kube-api-access-4mh7d\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:22:46.931184 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:46.930941    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a956fba-d8ca-4732-a017-dcbcddca6aa9-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:22:47.020558 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:47.020478    4942 generic.go:356] "Generic (PLEG): container finished" podID="3a956fba-d8ca-4732-a017-dcbcddca6aa9" containerID="1ce510f79b8f36beda00cff03a18a3292e5ee41c20eaf3fbe98af9731d83a7ab" exitCode=0
Nov 02 12:22:47.020558 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:47.020526    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"
Nov 02 12:22:47.020741 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:47.020559    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z" event={"ID":"3a956fba-d8ca-4732-a017-dcbcddca6aa9","Type":"ContainerDied","Data":"1ce510f79b8f36beda00cff03a18a3292e5ee41c20eaf3fbe98af9731d83a7ab"}
Nov 02 12:22:47.020741 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:47.020622    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z" event={"ID":"3a956fba-d8ca-4732-a017-dcbcddca6aa9","Type":"ContainerDied","Data":"c18cd71d8235b650047a8d8b3ab5b2040cfcd61125dbf0833253415d7ddf7bb9"}
Nov 02 12:22:47.020741 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:47.020647    4942 scope.go:122] "RemoveContainer" containerID="1ce510f79b8f36beda00cff03a18a3292e5ee41c20eaf3fbe98af9731d83a7ab"
Nov 02 12:22:47.032586 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:47.032560    4942 scope.go:122] "RemoveContainer" containerID="1ce510f79b8f36beda00cff03a18a3292e5ee41c20eaf3fbe98af9731d83a7ab"
Nov 02 12:22:47.032958 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:22:47.032931    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ce510f79b8f36beda00cff03a18a3292e5ee41c20eaf3fbe98af9731d83a7ab\": container with ID starting with 1ce510f79b8f36beda00cff03a18a3292e5ee41c20eaf3fbe98af9731d83a7ab not found: ID does not exist" containerID="1ce510f79b8f36beda00cff03a18a3292e5ee41c20eaf3fbe98af9731d83a7ab"
Nov 02 12:22:47.033340 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:47.033273    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ce510f79b8f36beda00cff03a18a3292e5ee41c20eaf3fbe98af9731d83a7ab"} err="failed to get container status \"1ce510f79b8f36beda00cff03a18a3292e5ee41c20eaf3fbe98af9731d83a7ab\": rpc error: code = NotFound desc = could not find container \"1ce510f79b8f36beda00cff03a18a3292e5ee41c20eaf3fbe98af9731d83a7ab\": container with ID starting with 1ce510f79b8f36beda00cff03a18a3292e5ee41c20eaf3fbe98af9731d83a7ab not found: ID does not exist"
Nov 02 12:22:47.185771 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:47.185742    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"]
Nov 02 12:22:47.189989 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:47.189961    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fbf1444a-59fbc8bdd7-rq98z"]
Nov 02 12:22:47.543584 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:47.543548    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a956fba-d8ca-4732-a017-dcbcddca6aa9" path="/var/lib/kubelet/pods/3a956fba-d8ca-4732-a017-dcbcddca6aa9/volumes"
Nov 02 12:22:50.909199 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:50.909162    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"]
Nov 02 12:22:50.909796 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:50.909779    4942 cpu_manager.go:414] "containerMap: removing container" podUID="3a956fba-d8ca-4732-a017-dcbcddca6aa9" containerName="maas-api"
Nov 02 12:22:50.909835 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:50.909800    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a956fba-d8ca-4732-a017-dcbcddca6aa9" containerName="maas-api"
Nov 02 12:22:50.909912 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:50.909902    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a956fba-d8ca-4732-a017-dcbcddca6aa9" containerName="maas-api"
Nov 02 12:22:50.914409 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:50.914390    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:50.970881 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:50.970850    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-1cc3a7b3-gw-openshift-default-dockercfg-6xmlw\""
Nov 02 12:22:50.985440 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:50.984627    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8373905-a9ab-473e-a698-2e699bb89e33-istio-podinfo\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:50.985440 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:50.984795    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-istio-envoy\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:50.985440 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:50.984838    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-istio-data\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:50.985440 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:50.984862    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55jls\" (UniqueName: \"kubernetes.io/projected/b8373905-a9ab-473e-a698-2e699bb89e33-kube-api-access-55jls\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:50.985440 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:50.984967    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-credential-socket\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:50.985440 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:50.985012    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8373905-a9ab-473e-a698-2e699bb89e33-istiod-ca-cert\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:50.985440 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:50.985093    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-workload-socket\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:50.985440 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:50.985182    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8373905-a9ab-473e-a698-2e699bb89e33-istio-token\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:50.985440 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:50.985344    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-workload-certs\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.020818 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.020782    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"]
Nov 02 12:22:51.086532 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.086480    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8373905-a9ab-473e-a698-2e699bb89e33-istio-podinfo\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.086736 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.086641    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-istio-envoy\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.086736 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.086680    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-istio-data\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.086736 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.086704    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55jls\" (UniqueName: \"kubernetes.io/projected/b8373905-a9ab-473e-a698-2e699bb89e33-kube-api-access-55jls\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.086919 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.086735    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-credential-socket\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.086919 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.086768    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8373905-a9ab-473e-a698-2e699bb89e33-istiod-ca-cert\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.086919 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.086801    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-workload-socket\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.086919 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.086842    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8373905-a9ab-473e-a698-2e699bb89e33-istio-token\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.087269 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.086948    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-workload-certs\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.087463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.087435    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-workload-certs\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.087606 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.087464    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-istio-data\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.087737 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.087691    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-credential-socket\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.088286 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.088098    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-workload-socket\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.088487 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.088467    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8373905-a9ab-473e-a698-2e699bb89e33-istiod-ca-cert\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.089616 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.089592    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8373905-a9ab-473e-a698-2e699bb89e33-istio-podinfo\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.089808 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.089791    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-istio-envoy\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.148107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.148070    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8373905-a9ab-473e-a698-2e699bb89e33-istio-token\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.148252 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.148196    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55jls\" (UniqueName: \"kubernetes.io/projected/b8373905-a9ab-473e-a698-2e699bb89e33-kube-api-access-55jls\") pod \"e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") " pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.227901 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.227810    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:51.675493 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.675471    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"]
Nov 02 12:22:51.677366 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:22:51.677342    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8373905_a9ab_473e_a698_2e699bb89e33.slice/crio-cb2ae09e8ec9d28e0544392afca6e94a4b72aa0e55d8d451b6665a80b690e0ef WatchSource:0}: Error finding container cb2ae09e8ec9d28e0544392afca6e94a4b72aa0e55d8d451b6665a80b690e0ef: Status 404 returned error can't find the container with id cb2ae09e8ec9d28e0544392afca6e94a4b72aa0e55d8d451b6665a80b690e0ef
Nov 02 12:22:51.680290 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.680253    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:22:51.680419 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.680346    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:22:51.680419 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:51.680400    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:22:52.061821 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:52.061731    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm" event={"ID":"b8373905-a9ab-473e-a698-2e699bb89e33","Type":"ContainerStarted","Data":"06cee18cee8c69fa56a87b1444a1107d37db278144e418c64ab1f6c173ca8ccc"}
Nov 02 12:22:52.061821 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:52.061791    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm" event={"ID":"b8373905-a9ab-473e-a698-2e699bb89e33","Type":"ContainerStarted","Data":"cb2ae09e8ec9d28e0544392afca6e94a4b72aa0e55d8d451b6665a80b690e0ef"}
Nov 02 12:22:52.182455 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:52.182396    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm" podStartSLOduration=2.182383442 podStartE2EDuration="2.182383442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:22:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:22:52.181219449 +0000 UTC m=+1598.752569206" watchObservedRunningTime="2026-11-02 12:22:52.182383442 +0000 UTC m=+1598.753733199"
Nov 02 12:22:52.228120 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:52.228077    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:52.234167 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:52.234141    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:53.094343 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:53.092446    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:22:53.094343 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:22:53.094239    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:23:00.986760 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:23:00.986719    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:23:05.166075 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.166038    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"]
Nov 02 12:23:05.170260 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.170241    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.222478 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.222445    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-c5ldd\""
Nov 02 12:23:05.243991 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.243931    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.244368 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.244338    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.244623 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.244601    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.244712 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.244643    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.244772 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.244716    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.244772 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.244748    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.244868 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.244786    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.244868 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.244838    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.245022 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.244896    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb8rs\" (UniqueName: \"kubernetes.io/projected/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-kube-api-access-wb8rs\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.275744 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.275705    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"]
Nov 02 12:23:05.346029 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.345992    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.346177 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.346065    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.346177 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.346098    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.346177 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.346166    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.346282 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.346198    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.346282 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.346243    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.346396 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.346317    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.346518 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.346493    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.346624 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.346515    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb8rs\" (UniqueName: \"kubernetes.io/projected/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-kube-api-access-wb8rs\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.346624 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.346533    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.346741 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.346638    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.346799 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.346751    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.346848 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.346796    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.346982 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.346959    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.348715 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.348686    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.348715 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.348693    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.424397 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.409111    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.424397 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.419252    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb8rs\" (UniqueName: \"kubernetes.io/projected/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-kube-api-access-wb8rs\") pod \"maas-default-gateway-openshift-default-6dcf757899-62r9f\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.485210 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.485176    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:05.921553 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.921513    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"]
Nov 02 12:23:05.922525 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:23:05.922496    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1ee0c8b_9f5a_4e8c_b468_a0eb7b139ffb.slice/crio-64e2f4328c8474a3ae74c82bad7bd0063962e0f93d97960f5aece30052fe6c82 WatchSource:0}: Error finding container 64e2f4328c8474a3ae74c82bad7bd0063962e0f93d97960f5aece30052fe6c82: Status 404 returned error can't find the container with id 64e2f4328c8474a3ae74c82bad7bd0063962e0f93d97960f5aece30052fe6c82
Nov 02 12:23:05.924852 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.924815    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:23:05.924967 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.924901    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:23:05.924967 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:05.924947    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:23:06.228267 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:06.228182    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f" event={"ID":"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb","Type":"ContainerStarted","Data":"c244b9bbd4f49229f0e9d85ffc4ee3fd4e6445e1b1846a35437fe0b3d516ebdb"}
Nov 02 12:23:06.228267 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:06.228247    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f" event={"ID":"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb","Type":"ContainerStarted","Data":"64e2f4328c8474a3ae74c82bad7bd0063962e0f93d97960f5aece30052fe6c82"}
Nov 02 12:23:06.350065 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:06.350011    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f" podStartSLOduration=1.349996549 podStartE2EDuration="1.349996549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:23:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:23:06.34733291 +0000 UTC m=+1612.918682668" watchObservedRunningTime="2026-11-02 12:23:06.349996549 +0000 UTC m=+1612.921346306"
Nov 02 12:23:06.485679 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:06.485585    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:06.489763 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:06.489406    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-62r9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 12:23:06.489763 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:06.489493    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f" podUID="e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 12:23:07.485913 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:07.485862    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-62r9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 12:23:07.486290 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:07.485943    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f" podUID="e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 12:23:08.486585 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:08.486545    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-62r9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 12:23:08.487079 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:08.486613    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f" podUID="e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 12:23:09.485818 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:09.485774    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-62r9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 12:23:09.486004 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:09.485845    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f" podUID="e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 12:23:10.487683 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:10.487645    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-62r9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 12:23:10.488194 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:10.487707    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f" podUID="e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 12:23:11.486400 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:11.486344    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-62r9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 12:23:11.486731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:11.486677    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f" podUID="e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 12:23:12.492322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:12.487404    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-62r9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 12:23:12.492322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:12.487463    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f" podUID="e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 12:23:13.486184 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:13.486142    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-62r9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Nov 02 12:23:13.486367 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:13.486212    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f" podUID="e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Nov 02 12:23:15.528444 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:15.528405    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-62r9f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 12:23:15.528962 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:15.528467    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f" podUID="e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:23:15.709569 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:15.709532    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:15.709924 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:15.709896    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:15.711042 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:15.711019    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:23:20.359601 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.359569    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"]
Nov 02 12:23:20.364264 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.364240    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.448952 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.448916    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"]
Nov 02 12:23:20.483573 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.483531    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.483755 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.483615    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn58f\" (UniqueName: \"kubernetes.io/projected/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-kube-api-access-bn58f\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.483755 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.483655    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.483755 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.483679    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.483755 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.483748    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.483982 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.483802    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.483982 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.483853    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.483982 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.483918    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.484289 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.484258    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.585143 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.585109    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.585143 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.585146    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.585381 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.585270    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.585381 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.585363    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.585457 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.585407    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.585457 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.585445    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.585571 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.585548    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.585636 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.585571    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.585636 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.585619    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.585750 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.585641    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.585750 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.585711    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bn58f\" (UniqueName: \"kubernetes.io/projected/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-kube-api-access-bn58f\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.585750 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.585732    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.585905 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.585854    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.585988 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.585969    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.587811 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.587787    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.588045 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.588028    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.646119 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.646061    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.646262 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.646143    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn58f\" (UniqueName: \"kubernetes.io/projected/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-kube-api-access-bn58f\") pod \"maas-default-gateway-openshift-default-6dcf757899-jmqgd\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:20.677113 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:20.677092    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:21.130107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:21.130074    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"]
Nov 02 12:23:21.130727 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:23:21.130703    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2e3be20_09b2_4bfc_8865_11cbb68d8e2a.slice/crio-4f82ba09b8a31184c4939304d13637c0f9a30a3333bc93bb5f4d9a7dc0b6af56 WatchSource:0}: Error finding container 4f82ba09b8a31184c4939304d13637c0f9a30a3333bc93bb5f4d9a7dc0b6af56: Status 404 returned error can't find the container with id 4f82ba09b8a31184c4939304d13637c0f9a30a3333bc93bb5f4d9a7dc0b6af56
Nov 02 12:23:21.133258 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:21.133220    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:23:21.133399 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:21.133338    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:23:21.133468 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:21.133397    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:23:21.351378 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:21.351346    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd" event={"ID":"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a","Type":"ContainerStarted","Data":"46db96d06b0bae9101aa712bb4b40b7cd408432cb5efd47b9eb936f957a3b7da"}
Nov 02 12:23:21.351545 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:21.351396    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd" event={"ID":"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a","Type":"ContainerStarted","Data":"4f82ba09b8a31184c4939304d13637c0f9a30a3333bc93bb5f4d9a7dc0b6af56"}
Nov 02 12:23:21.472060 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:21.471463    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd" podStartSLOduration=1.471444956 podStartE2EDuration="1.471444956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:23:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:23:21.471116336 +0000 UTC m=+1628.042466092" watchObservedRunningTime="2026-11-02 12:23:21.471444956 +0000 UTC m=+1628.042794707"
Nov 02 12:23:21.678459 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:21.678414    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:22.688358 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:22.686475    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-jmqgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 02 12:23:22.688995 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:22.688939    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd" podUID="d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 02 12:23:23.683698 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:23.683662    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-jmqgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 02 12:23:23.683912 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:23.683723    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd" podUID="d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 02 12:23:24.015954 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:24.015875    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"]
Nov 02 12:23:24.677764 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:24.677725    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-jmqgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 02 12:23:24.678050 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:24.678020    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd" podUID="d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 02 12:23:25.684771 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:25.684734    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-jmqgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 02 12:23:25.685400 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:25.685367    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd" podUID="d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 02 12:23:26.679346 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:26.678640    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-jmqgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 02 12:23:26.679346 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:26.678706    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd" podUID="d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 02 12:23:27.678214 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:27.678178    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-jmqgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 02 12:23:27.678717 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:27.678239    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd" podUID="d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 02 12:23:28.677983 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:28.677939    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-jmqgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Nov 02 12:23:28.678269 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:28.678230    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd" podUID="d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Nov 02 12:23:30.719458 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:30.719413    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-jmqgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 12:23:30.719964 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:30.719485    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd" podUID="d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:23:30.986108 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:23:30.986026    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:23:31.083738 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:31.083691    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:31.084256 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:31.084215    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:31.085254 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:31.085233    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:23:35.475658 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.475623    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"]
Nov 02 12:23:35.479576 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.479557    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.616756 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.616711    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.616947 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.616840    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.617026 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.616953    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.617085 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.617020    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.617143 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.617109    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.617216 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.617197    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scb8t\" (UniqueName: \"kubernetes.io/projected/eea11872-8c21-4997-ab19-c9b81dbcf4fa-kube-api-access-scb8t\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.617275 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.617238    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.617275 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.617268    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.617400 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.617342    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.680658 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.680628    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"]
Nov 02 12:23:35.718559 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.718527    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.718744 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.718584    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.718744 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.718645    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.718744 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.718693    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.718974 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.718881    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.719035 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.718980    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.719035 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.718982    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scb8t\" (UniqueName: \"kubernetes.io/projected/eea11872-8c21-4997-ab19-c9b81dbcf4fa-kube-api-access-scb8t\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.719141 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.719048    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.719141 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.719073    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.719141 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.719104    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.719614 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.719589    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.719804 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.719678    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.719919 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.719867    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.719991 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.719960    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.721677 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.721632    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.721780 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.721765    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.779473 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.779415    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.779669 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.779647    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scb8t\" (UniqueName: \"kubernetes.io/projected/eea11872-8c21-4997-ab19-c9b81dbcf4fa-kube-api-access-scb8t\") pod \"maas-default-gateway-openshift-default-6dcf757899-7hsh2\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:35.793460 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:35.793434    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:36.226343 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:36.226312    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"]
Nov 02 12:23:36.226782 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:23:36.226756    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeea11872_8c21_4997_ab19_c9b81dbcf4fa.slice/crio-0021bc471661780e36f9faecf35587ff9f7d26dca49105d151b84ad768d090ca WatchSource:0}: Error finding container 0021bc471661780e36f9faecf35587ff9f7d26dca49105d151b84ad768d090ca: Status 404 returned error can't find the container with id 0021bc471661780e36f9faecf35587ff9f7d26dca49105d151b84ad768d090ca
Nov 02 12:23:36.229006 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:36.228974    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:23:36.229094 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:36.229041    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:23:36.229094 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:36.229068    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:23:36.508020 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:36.507928    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" event={"ID":"eea11872-8c21-4997-ab19-c9b81dbcf4fa","Type":"ContainerStarted","Data":"815996b2c876bde0ef3067780939afb791d433f7c977d5dbbf3ac039faf10826"}
Nov 02 12:23:36.510685 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:36.510649    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" event={"ID":"eea11872-8c21-4997-ab19-c9b81dbcf4fa","Type":"ContainerStarted","Data":"0021bc471661780e36f9faecf35587ff9f7d26dca49105d151b84ad768d090ca"}
Nov 02 12:23:36.627814 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:36.627763    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" podStartSLOduration=1.6277454470000001 podStartE2EDuration="1.627745447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:23:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:23:36.62666329 +0000 UTC m=+1643.198013061" watchObservedRunningTime="2026-11-02 12:23:36.627745447 +0000 UTC m=+1643.199095292"
Nov 02 12:23:36.799564 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:36.799462    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:37.796652 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:37.796608    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-7hsh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Nov 02 12:23:37.797134 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:37.796682    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Nov 02 12:23:38.794743 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:38.794699    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-7hsh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Nov 02 12:23:38.794902 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:38.794766    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Nov 02 12:23:39.036911 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.036867    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28" podUID="f3d8189b-983b-488a-9419-788fdf6f93cb" containerName="main" containerID="cri-o://f274bf9d845df772d3bcf9718fc8362009a999572727c257f3e90d9e4c838a5f" gracePeriod=45
Nov 02 12:23:39.390155 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.390121    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:23:39.491870 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.491832    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-model-cache\") pod \"f3d8189b-983b-488a-9419-788fdf6f93cb\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") "
Nov 02 12:23:39.492046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.491889    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-home\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-home\") pod \"f3d8189b-983b-488a-9419-788fdf6f93cb\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") "
Nov 02 12:23:39.492046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.491940    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f3d8189b-983b-488a-9419-788fdf6f93cb-kube-api-access-7ntd5\" (UniqueName: \"kubernetes.io/projected/f3d8189b-983b-488a-9419-788fdf6f93cb-kube-api-access-7ntd5\") pod \"f3d8189b-983b-488a-9419-788fdf6f93cb\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") "
Nov 02 12:23:39.492046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.492008    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-dshm\") pod \"f3d8189b-983b-488a-9419-788fdf6f93cb\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") "
Nov 02 12:23:39.492255 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.492069    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f3d8189b-983b-488a-9419-788fdf6f93cb-tls-certs\" (UniqueName: \"kubernetes.io/secret/f3d8189b-983b-488a-9419-788fdf6f93cb-tls-certs\") pod \"f3d8189b-983b-488a-9419-788fdf6f93cb\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") "
Nov 02 12:23:39.492255 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.492079    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-home" pod "f3d8189b-983b-488a-9419-788fdf6f93cb" (UID: "f3d8189b-983b-488a-9419-788fdf6f93cb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:23:39.492255 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.492102    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-tmp-dir\") pod \"f3d8189b-983b-488a-9419-788fdf6f93cb\" (UID: \"f3d8189b-983b-488a-9419-788fdf6f93cb\") "
Nov 02 12:23:39.492255 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.492100    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-model-cache" pod "f3d8189b-983b-488a-9419-788fdf6f93cb" (UID: "f3d8189b-983b-488a-9419-788fdf6f93cb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:23:39.492584 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.492554    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-tmp-dir" pod "f3d8189b-983b-488a-9419-788fdf6f93cb" (UID: "f3d8189b-983b-488a-9419-788fdf6f93cb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:23:39.492723 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.492696    4942 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-tmp-dir\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:23:39.492773 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.492720    4942 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-model-cache\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:23:39.492773 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.492748    4942 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-home\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:23:39.494245 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.494223    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3d8189b-983b-488a-9419-788fdf6f93cb-tls-certs" pod "f3d8189b-983b-488a-9419-788fdf6f93cb" (UID: "f3d8189b-983b-488a-9419-788fdf6f93cb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:23:39.494881 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.494864    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-dshm" pod "f3d8189b-983b-488a-9419-788fdf6f93cb" (UID: "f3d8189b-983b-488a-9419-788fdf6f93cb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:23:39.495034 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.495008    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3d8189b-983b-488a-9419-788fdf6f93cb-kube-api-access-7ntd5" pod "f3d8189b-983b-488a-9419-788fdf6f93cb" (UID: "f3d8189b-983b-488a-9419-788fdf6f93cb"). InnerVolumeSpecName "kube-api-access-7ntd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:23:39.537101 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.537061    4942 generic.go:356] "Generic (PLEG): container finished" podID="f3d8189b-983b-488a-9419-788fdf6f93cb" containerID="f274bf9d845df772d3bcf9718fc8362009a999572727c257f3e90d9e4c838a5f" exitCode=0
Nov 02 12:23:39.537282 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.537264    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"
Nov 02 12:23:39.546046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.545852    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28" event={"ID":"f3d8189b-983b-488a-9419-788fdf6f93cb","Type":"ContainerDied","Data":"f274bf9d845df772d3bcf9718fc8362009a999572727c257f3e90d9e4c838a5f"}
Nov 02 12:23:39.546046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.545889    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28" event={"ID":"f3d8189b-983b-488a-9419-788fdf6f93cb","Type":"ContainerDied","Data":"5ca6aeadca5f55b9ef65008e747aabeec2beabf8b11f1c63b904c32d0cc51fd8"}
Nov 02 12:23:39.546046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.545915    4942 scope.go:122] "RemoveContainer" containerID="f274bf9d845df772d3bcf9718fc8362009a999572727c257f3e90d9e4c838a5f"
Nov 02 12:23:39.590203 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.589936    4942 scope.go:122] "RemoveContainer" containerID="f274bf9d845df772d3bcf9718fc8362009a999572727c257f3e90d9e4c838a5f"
Nov 02 12:23:39.590499 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:23:39.590410    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f274bf9d845df772d3bcf9718fc8362009a999572727c257f3e90d9e4c838a5f\": container with ID starting with f274bf9d845df772d3bcf9718fc8362009a999572727c257f3e90d9e4c838a5f not found: ID does not exist" containerID="f274bf9d845df772d3bcf9718fc8362009a999572727c257f3e90d9e4c838a5f"
Nov 02 12:23:39.590671 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.590447    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f274bf9d845df772d3bcf9718fc8362009a999572727c257f3e90d9e4c838a5f"} err="failed to get container status \"f274bf9d845df772d3bcf9718fc8362009a999572727c257f3e90d9e4c838a5f\": rpc error: code = NotFound desc = could not find container \"f274bf9d845df772d3bcf9718fc8362009a999572727c257f3e90d9e4c838a5f\": container with ID starting with f274bf9d845df772d3bcf9718fc8362009a999572727c257f3e90d9e4c838a5f not found: ID does not exist"
Nov 02 12:23:39.594000 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.593977    4942 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3d8189b-983b-488a-9419-788fdf6f93cb-tls-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:23:39.594138 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.594123    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ntd5\" (UniqueName: \"kubernetes.io/projected/f3d8189b-983b-488a-9419-788fdf6f93cb-kube-api-access-7ntd5\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:23:39.594231 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.594219    4942 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3d8189b-983b-488a-9419-788fdf6f93cb-dshm\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:23:39.705353 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.705316    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"]
Nov 02 12:23:39.709466 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.709433    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28"]
Nov 02 12:23:39.793838 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.793801    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-7hsh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Nov 02 12:23:39.794027 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:39.793863    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Nov 02 12:23:40.286016 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:40.285964    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w4-03c4e8/rate-test-model-03c4e8-kserve-7b8ddfd97c-b2s28" podUID="f3d8189b-983b-488a-9419-788fdf6f93cb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:23:40.794484 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:40.794449    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-7hsh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Nov 02 12:23:40.794675 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:40.794505    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Nov 02 12:23:41.542860 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:41.542826    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f3d8189b-983b-488a-9419-788fdf6f93cb" path="/var/lib/kubelet/pods/f3d8189b-983b-488a-9419-788fdf6f93cb/volumes"
Nov 02 12:23:41.794615 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:41.794529    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-7hsh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Nov 02 12:23:41.794615 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:41.794597    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Nov 02 12:23:42.794317 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:42.794268    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-7hsh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Nov 02 12:23:42.794836 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:42.794348    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Nov 02 12:23:43.794748 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:43.794707    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-7hsh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Nov 02 12:23:43.795130 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:43.794787    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Nov 02 12:23:44.013415 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:44.013380    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/ai-tenant-e2e-extmod-b-3a78b4_payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2_18798180-d3f0-4a87-8226-1b66d45a76bc/payload-processing/0.log"
Nov 02 12:23:45.836446 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:45.836403    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-7hsh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 12:23:45.836970 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:45.836465    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:23:46.629498 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:46.629461    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:46.629991 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:46.629971    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:46.631199 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:46.631178    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 12:23:54.106232 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:54.105062    4942 patch_prober.go:29] interesting pod/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 12:23:54.106232 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:54.105171    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm" podUID="b8373905-a9ab-473e-a698-2e699bb89e33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:23:54.670615 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:54.670584    4942 generic.go:356] "Generic (PLEG): container finished" podID="b8373905-a9ab-473e-a698-2e699bb89e33" containerID="06cee18cee8c69fa56a87b1444a1107d37db278144e418c64ab1f6c173ca8ccc" exitCode=0
Nov 02 12:23:54.670791 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:54.670658    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm" event={"ID":"b8373905-a9ab-473e-a698-2e699bb89e33","Type":"ContainerDied","Data":"06cee18cee8c69fa56a87b1444a1107d37db278144e418c64ab1f6c173ca8ccc"}
Nov 02 12:23:54.671099 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:54.671075    4942 scope.go:122] "RemoveContainer" containerID="06cee18cee8c69fa56a87b1444a1107d37db278144e418c64ab1f6c173ca8ccc"
Nov 02 12:23:54.672742 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:54.672701    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:23:54.672834 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:54.672816    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:23:54.672893 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:54.672876    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:23:55.710786 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:55.710748    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm" event={"ID":"b8373905-a9ab-473e-a698-2e699bb89e33","Type":"ContainerStarted","Data":"55441bb73a19d4cf3f60f20fe501a7fa6a35270cfabb629953813b0807f1f5ae"}
Nov 02 12:23:56.228869 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:56.228828    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:23:56.229090 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:56.229062    4942 patch_prober.go:29] interesting pod/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 02 12:23:56.229152 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:56.229134    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm" podUID="b8373905-a9ab-473e-a698-2e699bb89e33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 02 12:23:57.228906 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:57.228861    4942 patch_prober.go:29] interesting pod/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 02 12:23:57.229521 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:57.228927    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm" podUID="b8373905-a9ab-473e-a698-2e699bb89e33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 02 12:23:58.228826 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:58.228790    4942 patch_prober.go:29] interesting pod/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 02 12:23:58.228999 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:58.228844    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm" podUID="b8373905-a9ab-473e-a698-2e699bb89e33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 02 12:23:59.228963 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:59.228926    4942 patch_prober.go:29] interesting pod/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 02 12:23:59.229149 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:23:59.228985    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm" podUID="b8373905-a9ab-473e-a698-2e699bb89e33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 02 12:24:00.228422 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:00.228385    4942 patch_prober.go:29] interesting pod/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Nov 02 12:24:00.228597 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:00.228442    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm" podUID="b8373905-a9ab-473e-a698-2e699bb89e33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Nov 02 12:24:00.992524 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:24:00.992487    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:24:01.234066 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:01.234036    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:24:01.234363 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:01.234338    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:24:01.235380 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:01.235350    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:24:03.205492 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.205457    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"]
Nov 02 12:24:03.206034 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.206018    4942 cpu_manager.go:414] "containerMap: removing container" podUID="f3d8189b-983b-488a-9419-788fdf6f93cb" containerName="main"
Nov 02 12:24:03.206076 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.206036    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f3d8189b-983b-488a-9419-788fdf6f93cb" containerName="main"
Nov 02 12:24:03.206159 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.206150    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="f3d8189b-983b-488a-9419-788fdf6f93cb" containerName="main"
Nov 02 12:24:03.209542 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.209525    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.277472 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.277442    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-dshm\") pod \"sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.277639 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.277511    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-home\") pod \"sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.277639 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.277560    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twlsr\" (UniqueName: \"kubernetes.io/projected/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-kube-api-access-twlsr\") pod \"sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.277639 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.277629    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-tmp-dir\") pod \"sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.277786 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.277662    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-tls-certs\") pod \"sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.277871 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.277841    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-model-cache\") pod \"sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.312624 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.312596    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w4-dd087a\"/\"default-dockercfg-s8lc4\""
Nov 02 12:24:03.312762 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.312664    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w4-dd087a\"/\"kube-root-ca.crt\""
Nov 02 12:24:03.312762 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.312705    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w4-dd087a\"/\"openshift-service-ca.crt\""
Nov 02 12:24:03.312943 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.312929    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w4-dd087a\"/\"sub-test-model-dd087a-kserve-self-signed-certs\""
Nov 02 12:24:03.316108 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.316090    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"]
Nov 02 12:24:03.379091 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.379053    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-tmp-dir\") pod \"sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.379274 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.379120    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-tls-certs\") pod \"sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.379613 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.379587    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-tmp-dir\") pod \"sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.379795 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.379767    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-model-cache\") pod \"sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.379930 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.379906    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-dshm\") pod \"sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.380001 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.379986    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-home\") pod \"sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.380062 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.380016    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twlsr\" (UniqueName: \"kubernetes.io/projected/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-kube-api-access-twlsr\") pod \"sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.380143 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.380124    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-model-cache\") pod \"sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.380387 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.380364    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-home\") pod \"sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.383890 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.383792    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-dshm\") pod \"sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.384122 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.383937    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-tls-certs\") pod \"sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.440763 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.440733    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twlsr\" (UniqueName: \"kubernetes.io/projected/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-kube-api-access-twlsr\") pod \"sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") " pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.525834 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.525751    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:03.922203 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:03.922177    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"]
Nov 02 12:24:03.922849 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:24:03.922818    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc96d0961_c5a0_4cdc_a6fb_21f0a75329df.slice/crio-c62ddba983bf40511e784a742b452ba8b9b0ccda3a7b3b1ca2f87db32ce9d925 WatchSource:0}: Error finding container c62ddba983bf40511e784a742b452ba8b9b0ccda3a7b3b1ca2f87db32ce9d925: Status 404 returned error can't find the container with id c62ddba983bf40511e784a742b452ba8b9b0ccda3a7b3b1ca2f87db32ce9d925
Nov 02 12:24:04.200420 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.200344    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"]
Nov 02 12:24:04.200565 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.200530    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw" podUID="2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a" containerName="maas-api" containerID="cri-o://43abfe8cb138fd9445cccace1ca1798b9aa77e139cc4475997e235bbcad080e4" gracePeriod=30
Nov 02 12:24:04.492119 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.492091    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"
Nov 02 12:24:04.596330 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.596267    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-maas-api-tls\") pod \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\" (UID: \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\") "
Nov 02 12:24:04.596560 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.596539    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-kube-api-access-r6pdw\" (UniqueName: \"kubernetes.io/projected/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-kube-api-access-r6pdw\") pod \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\" (UID: \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\") "
Nov 02 12:24:04.596640 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.596582    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-metrics-certs\") pod \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\" (UID: \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\") "
Nov 02 12:24:04.596700 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.596658    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-openshift-service-ca\") pod \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\" (UID: \"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a\") "
Nov 02 12:24:04.597138 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.597105    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-openshift-service-ca" pod "2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a" (UID: "2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:24:04.598582 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.598553    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-maas-api-tls" pod "2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a" (UID: "2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:24:04.598688 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.598575    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-kube-api-access-r6pdw" pod "2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a" (UID: "2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a"). InnerVolumeSpecName "kube-api-access-r6pdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:24:04.598688 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.598678    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-metrics-certs" pod "2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a" (UID: "2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:24:04.698189 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.698153    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6pdw\" (UniqueName: \"kubernetes.io/projected/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-kube-api-access-r6pdw\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:24:04.698189 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.698187    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:24:04.698189 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.698198    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:24:04.698449 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.698206    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:24:04.842951 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.842856    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4" event={"ID":"c96d0961-c5a0-4cdc-a6fb-21f0a75329df","Type":"ContainerStarted","Data":"4639b5c3298cc9df0cbc4eaa33e3c713f506bd95fd7bb4fee098db22518b1c89"}
Nov 02 12:24:04.842951 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.842910    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4" event={"ID":"c96d0961-c5a0-4cdc-a6fb-21f0a75329df","Type":"ContainerStarted","Data":"c62ddba983bf40511e784a742b452ba8b9b0ccda3a7b3b1ca2f87db32ce9d925"}
Nov 02 12:24:04.844523 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.844495    4942 generic.go:356] "Generic (PLEG): container finished" podID="2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a" containerID="43abfe8cb138fd9445cccace1ca1798b9aa77e139cc4475997e235bbcad080e4" exitCode=0
Nov 02 12:24:04.844664 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.844533    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"
Nov 02 12:24:04.844664 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.844567    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw" event={"ID":"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a","Type":"ContainerDied","Data":"43abfe8cb138fd9445cccace1ca1798b9aa77e139cc4475997e235bbcad080e4"}
Nov 02 12:24:04.844664 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.844610    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw" event={"ID":"2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a","Type":"ContainerDied","Data":"199e7675c970228e0d8ce8ecdf660057bb99867436d121a0200e679fde653c34"}
Nov 02 12:24:04.844664 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.844639    4942 scope.go:122] "RemoveContainer" containerID="43abfe8cb138fd9445cccace1ca1798b9aa77e139cc4475997e235bbcad080e4"
Nov 02 12:24:04.856939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.856919    4942 scope.go:122] "RemoveContainer" containerID="43abfe8cb138fd9445cccace1ca1798b9aa77e139cc4475997e235bbcad080e4"
Nov 02 12:24:04.857239 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:24:04.857218    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43abfe8cb138fd9445cccace1ca1798b9aa77e139cc4475997e235bbcad080e4\": container with ID starting with 43abfe8cb138fd9445cccace1ca1798b9aa77e139cc4475997e235bbcad080e4 not found: ID does not exist" containerID="43abfe8cb138fd9445cccace1ca1798b9aa77e139cc4475997e235bbcad080e4"
Nov 02 12:24:04.857293 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.857249    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43abfe8cb138fd9445cccace1ca1798b9aa77e139cc4475997e235bbcad080e4"} err="failed to get container status \"43abfe8cb138fd9445cccace1ca1798b9aa77e139cc4475997e235bbcad080e4\": rpc error: code = NotFound desc = could not find container \"43abfe8cb138fd9445cccace1ca1798b9aa77e139cc4475997e235bbcad080e4\": container with ID starting with 43abfe8cb138fd9445cccace1ca1798b9aa77e139cc4475997e235bbcad080e4 not found: ID does not exist"
Nov 02 12:24:04.962272 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:04.962221    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4" podStartSLOduration=1.962207533 podStartE2EDuration="1.962207533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:24:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:24:04.958870249 +0000 UTC m=+1671.530220007" watchObservedRunningTime="2026-11-02 12:24:04.962207533 +0000 UTC m=+1671.533557290"
Nov 02 12:24:05.071120 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:05.071084    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"]
Nov 02 12:24:05.077339 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:05.077311    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6a432197-79f4f6646c-rc7cw"]
Nov 02 12:24:05.526363 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:05.526329    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:05.543637 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:05.543610    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a" path="/var/lib/kubelet/pods/2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a/volumes"
Nov 02 12:24:13.526667 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:13.526617    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:13.543642 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:13.543618    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:13.994728 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:13.994703    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:24:14.069657 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.069627    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"]
Nov 02 12:24:14.070183 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.070169    4942 cpu_manager.go:414] "containerMap: removing container" podUID="2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a" containerName="maas-api"
Nov 02 12:24:14.070242 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.070183    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a" containerName="maas-api"
Nov 02 12:24:14.070293 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.070284    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e2c7ff8-8b9a-4bcf-9b7c-418ecf231b4a" containerName="maas-api"
Nov 02 12:24:14.075167 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.075149    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"
Nov 02 12:24:14.130475 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.130433    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-6278d2f81b78\""
Nov 02 12:24:14.137238 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.137212    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"]
Nov 02 12:24:14.171349 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.168550    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Nov 02 12:24:14.171349 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.168642    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Nov 02 12:24:14.171349 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.168688    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Nov 02 12:24:14.171349 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.168719    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-fkwpc\""
Nov 02 12:24:14.171349 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.168736    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Nov 02 12:24:14.232082 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.232049    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b632b473-36e0-4dcf-908e-722392bb1a6f-routing-overlay-volume\") pod \"payload-processing-external-model-7bc9c479f4-zpmgw\" (UID: \"b632b473-36e0-4dcf-908e-722392bb1a6f\") " pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"
Nov 02 12:24:14.232265 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.232123    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdc4v\" (UniqueName: \"kubernetes.io/projected/b632b473-36e0-4dcf-908e-722392bb1a6f-kube-api-access-cdc4v\") pod \"payload-processing-external-model-7bc9c479f4-zpmgw\" (UID: \"b632b473-36e0-4dcf-908e-722392bb1a6f\") " pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"
Nov 02 12:24:14.232265 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.232225    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/b632b473-36e0-4dcf-908e-722392bb1a6f-provider-credentials\") pod \"payload-processing-external-model-7bc9c479f4-zpmgw\" (UID: \"b632b473-36e0-4dcf-908e-722392bb1a6f\") " pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"
Nov 02 12:24:14.232421 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.232374    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b632b473-36e0-4dcf-908e-722392bb1a6f-plugins-config-volume\") pod \"payload-processing-external-model-7bc9c479f4-zpmgw\" (UID: \"b632b473-36e0-4dcf-908e-722392bb1a6f\") " pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"
Nov 02 12:24:14.333221 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.333118    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b632b473-36e0-4dcf-908e-722392bb1a6f-plugins-config-volume\") pod \"payload-processing-external-model-7bc9c479f4-zpmgw\" (UID: \"b632b473-36e0-4dcf-908e-722392bb1a6f\") " pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"
Nov 02 12:24:14.333442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.333224    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b632b473-36e0-4dcf-908e-722392bb1a6f-routing-overlay-volume\") pod \"payload-processing-external-model-7bc9c479f4-zpmgw\" (UID: \"b632b473-36e0-4dcf-908e-722392bb1a6f\") " pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"
Nov 02 12:24:14.333442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.333332    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdc4v\" (UniqueName: \"kubernetes.io/projected/b632b473-36e0-4dcf-908e-722392bb1a6f-kube-api-access-cdc4v\") pod \"payload-processing-external-model-7bc9c479f4-zpmgw\" (UID: \"b632b473-36e0-4dcf-908e-722392bb1a6f\") " pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"
Nov 02 12:24:14.333442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.333401    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/b632b473-36e0-4dcf-908e-722392bb1a6f-provider-credentials\") pod \"payload-processing-external-model-7bc9c479f4-zpmgw\" (UID: \"b632b473-36e0-4dcf-908e-722392bb1a6f\") " pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"
Nov 02 12:24:14.334216 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.334185    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b632b473-36e0-4dcf-908e-722392bb1a6f-routing-overlay-volume\") pod \"payload-processing-external-model-7bc9c479f4-zpmgw\" (UID: \"b632b473-36e0-4dcf-908e-722392bb1a6f\") " pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"
Nov 02 12:24:14.334369 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.334239    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b632b473-36e0-4dcf-908e-722392bb1a6f-plugins-config-volume\") pod \"payload-processing-external-model-7bc9c479f4-zpmgw\" (UID: \"b632b473-36e0-4dcf-908e-722392bb1a6f\") " pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"
Nov 02 12:24:14.337738 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.337693    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/b632b473-36e0-4dcf-908e-722392bb1a6f-provider-credentials\") pod \"payload-processing-external-model-7bc9c479f4-zpmgw\" (UID: \"b632b473-36e0-4dcf-908e-722392bb1a6f\") " pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"
Nov 02 12:24:14.395648 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.395558    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdc4v\" (UniqueName: \"kubernetes.io/projected/b632b473-36e0-4dcf-908e-722392bb1a6f-kube-api-access-cdc4v\") pod \"payload-processing-external-model-7bc9c479f4-zpmgw\" (UID: \"b632b473-36e0-4dcf-908e-722392bb1a6f\") " pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"
Nov 02 12:24:14.692840 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:14.692809    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"
Nov 02 12:24:15.072212 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:15.072180    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"]
Nov 02 12:24:15.073274 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:24:15.073248    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb632b473_36e0_4dcf_908e_722392bb1a6f.slice/crio-d7ff6ac5ffa1f4518e83e7190c6a41e63a9395e0bfdfe476437c743d6d4974df WatchSource:0}: Error finding container d7ff6ac5ffa1f4518e83e7190c6a41e63a9395e0bfdfe476437c743d6d4974df: Status 404 returned error can't find the container with id d7ff6ac5ffa1f4518e83e7190c6a41e63a9395e0bfdfe476437c743d6d4974df
Nov 02 12:24:16.014910 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:16.013611    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw" event={"ID":"b632b473-36e0-4dcf-908e-722392bb1a6f","Type":"ContainerStarted","Data":"6ba33459e371624436b88ea26806babe8ca641e1dbf70fb6f9bc6c073b0b3fc7"}
Nov 02 12:24:16.014910 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:16.013684    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw" event={"ID":"b632b473-36e0-4dcf-908e-722392bb1a6f","Type":"ContainerStarted","Data":"d7ff6ac5ffa1f4518e83e7190c6a41e63a9395e0bfdfe476437c743d6d4974df"}
Nov 02 12:24:16.014910 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:16.014833    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"
Nov 02 12:24:16.134106 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:16.134046    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw" podStartSLOduration=2.134033605 podStartE2EDuration="2.134033605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:24:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:24:16.130628168 +0000 UTC m=+1682.701977926" watchObservedRunningTime="2026-11-02 12:24:16.134033605 +0000 UTC m=+1682.705383361"
Nov 02 12:24:24.287845 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:24.287811    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"]
Nov 02 12:24:24.290889 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:24.290855    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw" podUID="b632b473-36e0-4dcf-908e-722392bb1a6f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 12:24:26.892965 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:26.892921    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"]
Nov 02 12:24:26.896917 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:26.896893    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"
Nov 02 12:24:26.962357 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:26.959870    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g\" (UID: \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"
Nov 02 12:24:26.962357 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:26.960000    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g\" (UID: \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"
Nov 02 12:24:26.962357 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:26.960055    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55c74\" (UniqueName: \"kubernetes.io/projected/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-kube-api-access-55c74\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g\" (UID: \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"
Nov 02 12:24:26.962357 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:26.960145    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g\" (UID: \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"
Nov 02 12:24:26.999475 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:26.999445    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-3a78b4\"/\"agc-cred-dcd507907f10\""
Nov 02 12:24:27.004085 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:27.004060    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"]
Nov 02 12:24:27.061601 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:27.061556    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55c74\" (UniqueName: \"kubernetes.io/projected/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-kube-api-access-55c74\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g\" (UID: \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"
Nov 02 12:24:27.061766 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:27.061662    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g\" (UID: \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"
Nov 02 12:24:27.061834 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:27.061787    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g\" (UID: \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"
Nov 02 12:24:27.061900 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:27.061868    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g\" (UID: \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"
Nov 02 12:24:27.062715 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:27.062682    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g\" (UID: \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"
Nov 02 12:24:27.062848 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:27.062778    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g\" (UID: \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"
Nov 02 12:24:27.064720 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:27.064685    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g\" (UID: \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"
Nov 02 12:24:27.122683 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:27.122655    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55c74\" (UniqueName: \"kubernetes.io/projected/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-kube-api-access-55c74\") pod \"payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g\" (UID: \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\") " pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"
Nov 02 12:24:27.212421 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:27.212330    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"
Nov 02 12:24:27.590877 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:27.590845    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"]
Nov 02 12:24:27.592139 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:24:27.592101    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5793ce7_ecac_4c9e_ab91_c5d5723a187f.slice/crio-c8485d599084773b22d23034b1b3e09941e47ed632afdd935023b39dc133fcde WatchSource:0}: Error finding container c8485d599084773b22d23034b1b3e09941e47ed632afdd935023b39dc133fcde: Status 404 returned error can't find the container with id c8485d599084773b22d23034b1b3e09941e47ed632afdd935023b39dc133fcde
Nov 02 12:24:28.101870 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:28.101826    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g" event={"ID":"e5793ce7-ecac-4c9e-ab91-c5d5723a187f","Type":"ContainerStarted","Data":"4d5682393615b890c5eedc80af921907d40085bff5e8c6448123fddb7fbcd834"}
Nov 02 12:24:28.102231 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:28.101902    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g" event={"ID":"e5793ce7-ecac-4c9e-ab91-c5d5723a187f","Type":"ContainerStarted","Data":"c8485d599084773b22d23034b1b3e09941e47ed632afdd935023b39dc133fcde"}
Nov 02 12:24:28.102231 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:28.101931    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"
Nov 02 12:24:28.225372 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:28.225316    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g" podStartSLOduration=2.225285208 podStartE2EDuration="2.225285208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:24:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:24:28.222929737 +0000 UTC m=+1694.794279491" watchObservedRunningTime="2026-11-02 12:24:28.225285208 +0000 UTC m=+1694.796634964"
Nov 02 12:24:29.246844 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.246809    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"]
Nov 02 12:24:29.251277 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.251256    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.288191 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.288151    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw" podUID="b632b473-36e0-4dcf-908e-722392bb1a6f" containerName="payload-processing" containerID="cri-o://6ba33459e371624436b88ea26806babe8ca641e1dbf70fb6f9bc6c073b0b3fc7" gracePeriod=25
Nov 02 12:24:29.288926 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.288744    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.288926 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.288908    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.289041 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.288972    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.289084 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.289055    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.289189 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.289168    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlxmq\" (UniqueName: \"kubernetes.io/projected/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-kube-api-access-hlxmq\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.289267 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.289252    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.289407 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.289380    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.303636 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.303613    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 02 12:24:29.303746 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.303706    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 02 12:24:29.303811 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.303748    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 02 12:24:29.352601 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.352572    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-j8d6n\""
Nov 02 12:24:29.357865 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.357841    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"]
Nov 02 12:24:29.390404 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.390367    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.390573 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.390449    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.390573 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.390497    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.390726 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.390587    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlxmq\" (UniqueName: \"kubernetes.io/projected/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-kube-api-access-hlxmq\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.390726 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.390669    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.390726 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.390716    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.390976 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.390745    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.391660 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.391635    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.394610 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.394587    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.395077 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.395054    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.397112 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.397082    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.397461 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.397440    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.451710 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.451681    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlxmq\" (UniqueName: \"kubernetes.io/projected/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-kube-api-access-hlxmq\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.451849 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.451738    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-r577d\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.589534 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.576763    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:29.699751 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.699724    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"
Nov 02 12:24:29.796482 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.796441    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b632b473-36e0-4dcf-908e-722392bb1a6f-kube-api-access-cdc4v\" (UniqueName: \"kubernetes.io/projected/b632b473-36e0-4dcf-908e-722392bb1a6f-kube-api-access-cdc4v\") pod \"b632b473-36e0-4dcf-908e-722392bb1a6f\" (UID: \"b632b473-36e0-4dcf-908e-722392bb1a6f\") "
Nov 02 12:24:29.796647 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.796506    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b632b473-36e0-4dcf-908e-722392bb1a6f-provider-credentials\" (UniqueName: \"kubernetes.io/projected/b632b473-36e0-4dcf-908e-722392bb1a6f-provider-credentials\") pod \"b632b473-36e0-4dcf-908e-722392bb1a6f\" (UID: \"b632b473-36e0-4dcf-908e-722392bb1a6f\") "
Nov 02 12:24:29.796999 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.796966    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b632b473-36e0-4dcf-908e-722392bb1a6f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b632b473-36e0-4dcf-908e-722392bb1a6f-plugins-config-volume\") pod \"b632b473-36e0-4dcf-908e-722392bb1a6f\" (UID: \"b632b473-36e0-4dcf-908e-722392bb1a6f\") "
Nov 02 12:24:29.797277 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.797255    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b632b473-36e0-4dcf-908e-722392bb1a6f-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b632b473-36e0-4dcf-908e-722392bb1a6f-routing-overlay-volume\") pod \"b632b473-36e0-4dcf-908e-722392bb1a6f\" (UID: \"b632b473-36e0-4dcf-908e-722392bb1a6f\") "
Nov 02 12:24:29.797905 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.797482    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b632b473-36e0-4dcf-908e-722392bb1a6f-plugins-config-volume" pod "b632b473-36e0-4dcf-908e-722392bb1a6f" (UID: "b632b473-36e0-4dcf-908e-722392bb1a6f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:24:29.798008 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.797762    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b632b473-36e0-4dcf-908e-722392bb1a6f-routing-overlay-volume" pod "b632b473-36e0-4dcf-908e-722392bb1a6f" (UID: "b632b473-36e0-4dcf-908e-722392bb1a6f"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:24:29.798765 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.798740    4942 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b632b473-36e0-4dcf-908e-722392bb1a6f-plugins-config-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:24:29.798968 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.798887    4942 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/b632b473-36e0-4dcf-908e-722392bb1a6f-routing-overlay-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:24:29.799548 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.799508    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b632b473-36e0-4dcf-908e-722392bb1a6f-kube-api-access-cdc4v" pod "b632b473-36e0-4dcf-908e-722392bb1a6f" (UID: "b632b473-36e0-4dcf-908e-722392bb1a6f"). InnerVolumeSpecName "kube-api-access-cdc4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:24:29.806733 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.806703    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b632b473-36e0-4dcf-908e-722392bb1a6f-provider-credentials" pod "b632b473-36e0-4dcf-908e-722392bb1a6f" (UID: "b632b473-36e0-4dcf-908e-722392bb1a6f"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:24:29.900365 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.900293    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdc4v\" (UniqueName: \"kubernetes.io/projected/b632b473-36e0-4dcf-908e-722392bb1a6f-kube-api-access-cdc4v\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:24:29.900365 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:29.900354    4942 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/b632b473-36e0-4dcf-908e-722392bb1a6f-provider-credentials\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:24:30.103228 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:30.103193    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"]
Nov 02 12:24:30.104648 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:24:30.104620    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f4b8c8_2ce1_4d29_88f8_d36848c4e25b.slice/crio-deca0f20da734fadc8e5bdc8aa63244b075c5c3c9d6ddbe495d8eb5b8e6ecf04 WatchSource:0}: Error finding container deca0f20da734fadc8e5bdc8aa63244b075c5c3c9d6ddbe495d8eb5b8e6ecf04: Status 404 returned error can't find the container with id deca0f20da734fadc8e5bdc8aa63244b075c5c3c9d6ddbe495d8eb5b8e6ecf04
Nov 02 12:24:30.116631 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:30.116601    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d" event={"ID":"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b","Type":"ContainerStarted","Data":"deca0f20da734fadc8e5bdc8aa63244b075c5c3c9d6ddbe495d8eb5b8e6ecf04"}
Nov 02 12:24:30.118187 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:30.118162    4942 generic.go:356] "Generic (PLEG): container finished" podID="b632b473-36e0-4dcf-908e-722392bb1a6f" containerID="6ba33459e371624436b88ea26806babe8ca641e1dbf70fb6f9bc6c073b0b3fc7" exitCode=0
Nov 02 12:24:30.118284 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:30.118223    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"
Nov 02 12:24:30.118284 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:30.118248    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw" event={"ID":"b632b473-36e0-4dcf-908e-722392bb1a6f","Type":"ContainerDied","Data":"6ba33459e371624436b88ea26806babe8ca641e1dbf70fb6f9bc6c073b0b3fc7"}
Nov 02 12:24:30.118413 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:30.118314    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw" event={"ID":"b632b473-36e0-4dcf-908e-722392bb1a6f","Type":"ContainerDied","Data":"d7ff6ac5ffa1f4518e83e7190c6a41e63a9395e0bfdfe476437c743d6d4974df"}
Nov 02 12:24:30.118413 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:30.118346    4942 scope.go:122] "RemoveContainer" containerID="6ba33459e371624436b88ea26806babe8ca641e1dbf70fb6f9bc6c073b0b3fc7"
Nov 02 12:24:30.130081 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:30.129656    4942 scope.go:122] "RemoveContainer" containerID="6ba33459e371624436b88ea26806babe8ca641e1dbf70fb6f9bc6c073b0b3fc7"
Nov 02 12:24:30.130081 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:24:30.129948    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ba33459e371624436b88ea26806babe8ca641e1dbf70fb6f9bc6c073b0b3fc7\": container with ID starting with 6ba33459e371624436b88ea26806babe8ca641e1dbf70fb6f9bc6c073b0b3fc7 not found: ID does not exist" containerID="6ba33459e371624436b88ea26806babe8ca641e1dbf70fb6f9bc6c073b0b3fc7"
Nov 02 12:24:30.130081 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:30.129976    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ba33459e371624436b88ea26806babe8ca641e1dbf70fb6f9bc6c073b0b3fc7"} err="failed to get container status \"6ba33459e371624436b88ea26806babe8ca641e1dbf70fb6f9bc6c073b0b3fc7\": rpc error: code = NotFound desc = could not find container \"6ba33459e371624436b88ea26806babe8ca641e1dbf70fb6f9bc6c073b0b3fc7\": container with ID starting with 6ba33459e371624436b88ea26806babe8ca641e1dbf70fb6f9bc6c073b0b3fc7 not found: ID does not exist"
Nov 02 12:24:30.285366 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:30.284086    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"]
Nov 02 12:24:30.288094 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:30.288064    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-7bc9c479f4-zpmgw"]
Nov 02 12:24:31.082868 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:24:31.082831    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:24:31.545637 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:31.545604    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b632b473-36e0-4dcf-908e-722392bb1a6f" path="/var/lib/kubelet/pods/b632b473-36e0-4dcf-908e-722392bb1a6f/volumes"
Nov 02 12:24:36.474182 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:36.474139    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:24:36.474518 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:36.474230    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:24:37.210349 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:37.210281    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d" event={"ID":"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b","Type":"ContainerStarted","Data":"6bbbbe6434aef334315757c07df8bd27c0cdd17299af80f6e6c0c2b5aaaf3632"}
Nov 02 12:24:37.210542 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:37.210384    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:37.212972 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:37.212935    4942 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-r577d container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 12:24:37.213116 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:37.213012    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d" podUID="f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 12:24:37.420460 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:37.418269    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d" podStartSLOduration=2.052771993 podStartE2EDuration="8.418247242s" totalImagesPullingTime="6.365475249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:24:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:24:37.378871806 +0000 UTC m=+1703.950221564" watchObservedRunningTime="2026-11-02 12:24:37.418247242 +0000 UTC m=+1703.989597000"
Nov 02 12:24:38.217703 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:38.217668    4942 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-r577d container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 12:24:38.218093 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:38.217727    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d" podUID="f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 12:24:39.111064 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:39.111036    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"
Nov 02 12:24:39.280789 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:39.280757    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"]
Nov 02 12:24:41.217910 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:41.217883    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:24:44.281889 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:44.281836    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2" podUID="18798180-d3f0-4a87-8226-1b66d45a76bc" containerName="payload-processing" containerID="cri-o://8fc898b8c6b0e3b90309d3c2bf917b973eaa7f6af2f3df2e27c0a9803d070ca3" gracePeriod=25
Nov 02 12:24:44.567260 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:44.567236    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"
Nov 02 12:24:44.741159 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:44.741122    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/18798180-d3f0-4a87-8226-1b66d45a76bc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18798180-d3f0-4a87-8226-1b66d45a76bc-plugins-config-volume\") pod \"18798180-d3f0-4a87-8226-1b66d45a76bc\" (UID: \"18798180-d3f0-4a87-8226-1b66d45a76bc\") "
Nov 02 12:24:44.741383 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:44.741193    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18798180-d3f0-4a87-8226-1b66d45a76bc-provider-credentials\" (UniqueName: \"kubernetes.io/projected/18798180-d3f0-4a87-8226-1b66d45a76bc-provider-credentials\") pod \"18798180-d3f0-4a87-8226-1b66d45a76bc\" (UID: \"18798180-d3f0-4a87-8226-1b66d45a76bc\") "
Nov 02 12:24:44.741483 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:44.741419    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18798180-d3f0-4a87-8226-1b66d45a76bc-kube-api-access-mwtnc\" (UniqueName: \"kubernetes.io/projected/18798180-d3f0-4a87-8226-1b66d45a76bc-kube-api-access-mwtnc\") pod \"18798180-d3f0-4a87-8226-1b66d45a76bc\" (UID: \"18798180-d3f0-4a87-8226-1b66d45a76bc\") "
Nov 02 12:24:44.741528 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:44.741492    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/18798180-d3f0-4a87-8226-1b66d45a76bc-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/18798180-d3f0-4a87-8226-1b66d45a76bc-routing-overlay-volume\") pod \"18798180-d3f0-4a87-8226-1b66d45a76bc\" (UID: \"18798180-d3f0-4a87-8226-1b66d45a76bc\") "
Nov 02 12:24:44.741644 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:44.741602    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18798180-d3f0-4a87-8226-1b66d45a76bc-plugins-config-volume" pod "18798180-d3f0-4a87-8226-1b66d45a76bc" (UID: "18798180-d3f0-4a87-8226-1b66d45a76bc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:24:44.741912 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:44.741882    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18798180-d3f0-4a87-8226-1b66d45a76bc-routing-overlay-volume" pod "18798180-d3f0-4a87-8226-1b66d45a76bc" (UID: "18798180-d3f0-4a87-8226-1b66d45a76bc"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:24:44.742049 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:44.742028    4942 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/18798180-d3f0-4a87-8226-1b66d45a76bc-routing-overlay-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:24:44.742133 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:44.742054    4942 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18798180-d3f0-4a87-8226-1b66d45a76bc-plugins-config-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:24:44.743455 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:44.743436    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18798180-d3f0-4a87-8226-1b66d45a76bc-provider-credentials" pod "18798180-d3f0-4a87-8226-1b66d45a76bc" (UID: "18798180-d3f0-4a87-8226-1b66d45a76bc"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:24:44.743648 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:44.743631    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18798180-d3f0-4a87-8226-1b66d45a76bc-kube-api-access-mwtnc" pod "18798180-d3f0-4a87-8226-1b66d45a76bc" (UID: "18798180-d3f0-4a87-8226-1b66d45a76bc"). InnerVolumeSpecName "kube-api-access-mwtnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:24:44.843670 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:44.843585    4942 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/18798180-d3f0-4a87-8226-1b66d45a76bc-provider-credentials\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:24:44.843670 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:44.843613    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwtnc\" (UniqueName: \"kubernetes.io/projected/18798180-d3f0-4a87-8226-1b66d45a76bc-kube-api-access-mwtnc\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:24:45.269178 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:45.269139    4942 generic.go:356] "Generic (PLEG): container finished" podID="18798180-d3f0-4a87-8226-1b66d45a76bc" containerID="8fc898b8c6b0e3b90309d3c2bf917b973eaa7f6af2f3df2e27c0a9803d070ca3" exitCode=0
Nov 02 12:24:45.269371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:45.269188    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"
Nov 02 12:24:45.269371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:45.269198    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2" event={"ID":"18798180-d3f0-4a87-8226-1b66d45a76bc","Type":"ContainerDied","Data":"8fc898b8c6b0e3b90309d3c2bf917b973eaa7f6af2f3df2e27c0a9803d070ca3"}
Nov 02 12:24:45.269371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:45.269252    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2" event={"ID":"18798180-d3f0-4a87-8226-1b66d45a76bc","Type":"ContainerDied","Data":"04f285c1ad55b82b42a648502e8ad3b34607d0ae2a754948e500124cb72c1b49"}
Nov 02 12:24:45.269371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:45.269283    4942 scope.go:122] "RemoveContainer" containerID="8fc898b8c6b0e3b90309d3c2bf917b973eaa7f6af2f3df2e27c0a9803d070ca3"
Nov 02 12:24:45.280517 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:45.280494    4942 scope.go:122] "RemoveContainer" containerID="8fc898b8c6b0e3b90309d3c2bf917b973eaa7f6af2f3df2e27c0a9803d070ca3"
Nov 02 12:24:45.280819 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:24:45.280791    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fc898b8c6b0e3b90309d3c2bf917b973eaa7f6af2f3df2e27c0a9803d070ca3\": container with ID starting with 8fc898b8c6b0e3b90309d3c2bf917b973eaa7f6af2f3df2e27c0a9803d070ca3 not found: ID does not exist" containerID="8fc898b8c6b0e3b90309d3c2bf917b973eaa7f6af2f3df2e27c0a9803d070ca3"
Nov 02 12:24:45.280887 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:45.280831    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fc898b8c6b0e3b90309d3c2bf917b973eaa7f6af2f3df2e27c0a9803d070ca3"} err="failed to get container status \"8fc898b8c6b0e3b90309d3c2bf917b973eaa7f6af2f3df2e27c0a9803d070ca3\": rpc error: code = NotFound desc = could not find container \"8fc898b8c6b0e3b90309d3c2bf917b973eaa7f6af2f3df2e27c0a9803d070ca3\": container with ID starting with 8fc898b8c6b0e3b90309d3c2bf917b973eaa7f6af2f3df2e27c0a9803d070ca3 not found: ID does not exist"
Nov 02 12:24:45.434094 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:45.434059    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"]
Nov 02 12:24:45.483559 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:45.483533    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-8696z46f2"]
Nov 02 12:24:45.543107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:24:45.543042    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="18798180-d3f0-4a87-8226-1b66d45a76bc" path="/var/lib/kubelet/pods/18798180-d3f0-4a87-8226-1b66d45a76bc/volumes"
Nov 02 12:25:00.092558 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:00.092519    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"]
Nov 02 12:25:00.093103 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:00.092798    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm" podUID="b8373905-a9ab-473e-a698-2e699bb89e33" containerName="istio-proxy" containerID="cri-o://55441bb73a19d4cf3f60f20fe501a7fa6a35270cfabb629953813b0807f1f5ae" gracePeriod=30
Nov 02 12:25:00.985129 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:25:00.985085    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:25:01.237145 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:01.237048    4942 patch_prober.go:29] interesting pod/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 12:25:01.237145 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:01.237124    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm" podUID="b8373905-a9ab-473e-a698-2e699bb89e33" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 12:25:05.380747 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.380712    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:25:05.421039 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.420951    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8373905-a9ab-473e-a698-2e699bb89e33-istio-token\" (UniqueName: \"kubernetes.io/projected/b8373905-a9ab-473e-a698-2e699bb89e33-istio-token\") pod \"b8373905-a9ab-473e-a698-2e699bb89e33\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") "
Nov 02 12:25:05.421172 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.421128    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b8373905-a9ab-473e-a698-2e699bb89e33-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8373905-a9ab-473e-a698-2e699bb89e33-istio-podinfo\") pod \"b8373905-a9ab-473e-a698-2e699bb89e33\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") "
Nov 02 12:25:05.421172 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.421156    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8373905-a9ab-473e-a698-2e699bb89e33-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8373905-a9ab-473e-a698-2e699bb89e33-istiod-ca-cert\") pod \"b8373905-a9ab-473e-a698-2e699bb89e33\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") "
Nov 02 12:25:05.421248 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.421180    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-credential-socket\") pod \"b8373905-a9ab-473e-a698-2e699bb89e33\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") "
Nov 02 12:25:05.421248 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.421234    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8373905-a9ab-473e-a698-2e699bb89e33-kube-api-access-55jls\" (UniqueName: \"kubernetes.io/projected/b8373905-a9ab-473e-a698-2e699bb89e33-kube-api-access-55jls\") pod \"b8373905-a9ab-473e-a698-2e699bb89e33\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") "
Nov 02 12:25:05.421379 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.421351    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-workload-certs\") pod \"b8373905-a9ab-473e-a698-2e699bb89e33\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") "
Nov 02 12:25:05.421379 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.421371    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-istio-envoy\") pod \"b8373905-a9ab-473e-a698-2e699bb89e33\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") "
Nov 02 12:25:05.421478 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.421403    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-workload-socket\") pod \"b8373905-a9ab-473e-a698-2e699bb89e33\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") "
Nov 02 12:25:05.421478 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.421423    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-istio-data\") pod \"b8373905-a9ab-473e-a698-2e699bb89e33\" (UID: \"b8373905-a9ab-473e-a698-2e699bb89e33\") "
Nov 02 12:25:05.421586 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.421518    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-credential-socket" pod "b8373905-a9ab-473e-a698-2e699bb89e33" (UID: "b8373905-a9ab-473e-a698-2e699bb89e33"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:25:05.421586 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.421566    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8373905-a9ab-473e-a698-2e699bb89e33-istiod-ca-cert" pod "b8373905-a9ab-473e-a698-2e699bb89e33" (UID: "b8373905-a9ab-473e-a698-2e699bb89e33"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:25:05.421780 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.421758    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-workload-certs" pod "b8373905-a9ab-473e-a698-2e699bb89e33" (UID: "b8373905-a9ab-473e-a698-2e699bb89e33"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:25:05.421857 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.421810    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-workload-socket" pod "b8373905-a9ab-473e-a698-2e699bb89e33" (UID: "b8373905-a9ab-473e-a698-2e699bb89e33"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:25:05.421857 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.421823    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-istio-data" pod "b8373905-a9ab-473e-a698-2e699bb89e33" (UID: "b8373905-a9ab-473e-a698-2e699bb89e33"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:25:05.422090 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.422071    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:25:05.422158 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.422093    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:25:05.422158 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.422107    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:25:05.422158 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.422115    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8373905-a9ab-473e-a698-2e699bb89e33-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:25:05.422158 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.422124    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:25:05.423217 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.423193    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8373905-a9ab-473e-a698-2e699bb89e33-istio-token" pod "b8373905-a9ab-473e-a698-2e699bb89e33" (UID: "b8373905-a9ab-473e-a698-2e699bb89e33"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:25:05.423429 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.423407    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b8373905-a9ab-473e-a698-2e699bb89e33-istio-podinfo" pod "b8373905-a9ab-473e-a698-2e699bb89e33" (UID: "b8373905-a9ab-473e-a698-2e699bb89e33"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:25:05.423650 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.423632    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8373905-a9ab-473e-a698-2e699bb89e33-kube-api-access-55jls" pod "b8373905-a9ab-473e-a698-2e699bb89e33" (UID: "b8373905-a9ab-473e-a698-2e699bb89e33"). InnerVolumeSpecName "kube-api-access-55jls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:25:05.423738 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.423719    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-istio-envoy" pod "b8373905-a9ab-473e-a698-2e699bb89e33" (UID: "b8373905-a9ab-473e-a698-2e699bb89e33"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:25:05.494738 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.494711    4942 generic.go:356] "Generic (PLEG): container finished" podID="b8373905-a9ab-473e-a698-2e699bb89e33" containerID="55441bb73a19d4cf3f60f20fe501a7fa6a35270cfabb629953813b0807f1f5ae" exitCode=0
Nov 02 12:25:05.494870 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.494759    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"
Nov 02 12:25:05.494870 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.494801    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm" event={"ID":"b8373905-a9ab-473e-a698-2e699bb89e33","Type":"ContainerDied","Data":"55441bb73a19d4cf3f60f20fe501a7fa6a35270cfabb629953813b0807f1f5ae"}
Nov 02 12:25:05.494870 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.494862    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm" event={"ID":"b8373905-a9ab-473e-a698-2e699bb89e33","Type":"ContainerDied","Data":"cb2ae09e8ec9d28e0544392afca6e94a4b72aa0e55d8d451b6665a80b690e0ef"}
Nov 02 12:25:05.495049 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.494893    4942 scope.go:122] "RemoveContainer" containerID="55441bb73a19d4cf3f60f20fe501a7fa6a35270cfabb629953813b0807f1f5ae"
Nov 02 12:25:05.506694 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.506673    4942 scope.go:122] "RemoveContainer" containerID="06cee18cee8c69fa56a87b1444a1107d37db278144e418c64ab1f6c173ca8ccc"
Nov 02 12:25:05.518482 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.517355    4942 scope.go:122] "RemoveContainer" containerID="55441bb73a19d4cf3f60f20fe501a7fa6a35270cfabb629953813b0807f1f5ae"
Nov 02 12:25:05.518567 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:25:05.518499    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55441bb73a19d4cf3f60f20fe501a7fa6a35270cfabb629953813b0807f1f5ae\": container with ID starting with 55441bb73a19d4cf3f60f20fe501a7fa6a35270cfabb629953813b0807f1f5ae not found: ID does not exist" containerID="55441bb73a19d4cf3f60f20fe501a7fa6a35270cfabb629953813b0807f1f5ae"
Nov 02 12:25:05.518567 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.518535    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55441bb73a19d4cf3f60f20fe501a7fa6a35270cfabb629953813b0807f1f5ae"} err="failed to get container status \"55441bb73a19d4cf3f60f20fe501a7fa6a35270cfabb629953813b0807f1f5ae\": rpc error: code = NotFound desc = could not find container \"55441bb73a19d4cf3f60f20fe501a7fa6a35270cfabb629953813b0807f1f5ae\": container with ID starting with 55441bb73a19d4cf3f60f20fe501a7fa6a35270cfabb629953813b0807f1f5ae not found: ID does not exist"
Nov 02 12:25:05.518567 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.518559    4942 scope.go:122] "RemoveContainer" containerID="06cee18cee8c69fa56a87b1444a1107d37db278144e418c64ab1f6c173ca8ccc"
Nov 02 12:25:05.519499 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:25:05.519473    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06cee18cee8c69fa56a87b1444a1107d37db278144e418c64ab1f6c173ca8ccc\": container with ID starting with 06cee18cee8c69fa56a87b1444a1107d37db278144e418c64ab1f6c173ca8ccc not found: ID does not exist" containerID="06cee18cee8c69fa56a87b1444a1107d37db278144e418c64ab1f6c173ca8ccc"
Nov 02 12:25:05.519587 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.519509    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06cee18cee8c69fa56a87b1444a1107d37db278144e418c64ab1f6c173ca8ccc"} err="failed to get container status \"06cee18cee8c69fa56a87b1444a1107d37db278144e418c64ab1f6c173ca8ccc\": rpc error: code = NotFound desc = could not find container \"06cee18cee8c69fa56a87b1444a1107d37db278144e418c64ab1f6c173ca8ccc\": container with ID starting with 06cee18cee8c69fa56a87b1444a1107d37db278144e418c64ab1f6c173ca8ccc not found: ID does not exist"
Nov 02 12:25:05.523617 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.523595    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8373905-a9ab-473e-a698-2e699bb89e33-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:25:05.523674 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.523620    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8373905-a9ab-473e-a698-2e699bb89e33-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:25:05.523674 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.523631    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55jls\" (UniqueName: \"kubernetes.io/projected/b8373905-a9ab-473e-a698-2e699bb89e33-kube-api-access-55jls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:25:05.523674 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.523641    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8373905-a9ab-473e-a698-2e699bb89e33-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:25:05.671981 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.671897    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"]
Nov 02 12:25:05.675144 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:05.675119    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-1cc3a7b3-gw-openshift-default-b9f9fc5d5-wn6fm"]
Nov 02 12:25:07.546464 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:25:07.546430    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8373905-a9ab-473e-a698-2e699bb89e33" path="/var/lib/kubelet/pods/b8373905-a9ab-473e-a698-2e699bb89e33/volumes"
Nov 02 12:25:31.034534 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:25:31.034494    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:26:00.984262 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:26:00.984222    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:26:13.803526 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:26:13.803502    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:26:13.812935 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:26:13.812907    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:26:30.992863 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:26:30.991044    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:27:00.984501 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:27:00.984414    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:27:04.376859 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:04.376820    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"]
Nov 02 12:27:04.377333 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:04.377076    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd" podUID="6929f886-c638-4507-989e-74f2750c9bb6" containerName="istio-proxy" containerID="cri-o://f0414cfe9d263bfaacd530f704efd5b0095e1487b440b6b0535b90ba91abd520" gracePeriod=30
Nov 02 12:27:04.414238 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:04.414208    4942 patch_prober.go:29] interesting pod/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 12:27:04.414391 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:04.414284    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd" podUID="6929f886-c638-4507-989e-74f2750c9bb6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 12:27:09.656638 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.656569    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:27:09.781532 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.781488    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-credential-socket\") pod \"6929f886-c638-4507-989e-74f2750c9bb6\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") "
Nov 02 12:27:09.781713 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.781553    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-istio-data\") pod \"6929f886-c638-4507-989e-74f2750c9bb6\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") "
Nov 02 12:27:09.781713 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.781647    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6929f886-c638-4507-989e-74f2750c9bb6-istio-token\" (UniqueName: \"kubernetes.io/projected/6929f886-c638-4507-989e-74f2750c9bb6-istio-token\") pod \"6929f886-c638-4507-989e-74f2750c9bb6\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") "
Nov 02 12:27:09.781713 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.781680    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6929f886-c638-4507-989e-74f2750c9bb6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6929f886-c638-4507-989e-74f2750c9bb6-istio-podinfo\") pod \"6929f886-c638-4507-989e-74f2750c9bb6\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") "
Nov 02 12:27:09.781866 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.781791    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-istio-envoy\") pod \"6929f886-c638-4507-989e-74f2750c9bb6\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") "
Nov 02 12:27:09.781866 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.781816    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-credential-socket" pod "6929f886-c638-4507-989e-74f2750c9bb6" (UID: "6929f886-c638-4507-989e-74f2750c9bb6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:27:09.781866 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.781833    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-istio-data" pod "6929f886-c638-4507-989e-74f2750c9bb6" (UID: "6929f886-c638-4507-989e-74f2750c9bb6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:27:09.782027 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.781924    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6929f886-c638-4507-989e-74f2750c9bb6-kube-api-access-wd2hr\" (UniqueName: \"kubernetes.io/projected/6929f886-c638-4507-989e-74f2750c9bb6-kube-api-access-wd2hr\") pod \"6929f886-c638-4507-989e-74f2750c9bb6\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") "
Nov 02 12:27:09.782027 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.781968    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-workload-certs\") pod \"6929f886-c638-4507-989e-74f2750c9bb6\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") "
Nov 02 12:27:09.782027 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.782019    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6929f886-c638-4507-989e-74f2750c9bb6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6929f886-c638-4507-989e-74f2750c9bb6-istiod-ca-cert\") pod \"6929f886-c638-4507-989e-74f2750c9bb6\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") "
Nov 02 12:27:09.782193 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.782068    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-workload-socket\") pod \"6929f886-c638-4507-989e-74f2750c9bb6\" (UID: \"6929f886-c638-4507-989e-74f2750c9bb6\") "
Nov 02 12:27:09.782819 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.782356    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-workload-certs" pod "6929f886-c638-4507-989e-74f2750c9bb6" (UID: "6929f886-c638-4507-989e-74f2750c9bb6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:27:09.782819 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.782524    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-workload-socket" pod "6929f886-c638-4507-989e-74f2750c9bb6" (UID: "6929f886-c638-4507-989e-74f2750c9bb6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:27:09.782819 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.782609    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6929f886-c638-4507-989e-74f2750c9bb6-istiod-ca-cert" pod "6929f886-c638-4507-989e-74f2750c9bb6" (UID: "6929f886-c638-4507-989e-74f2750c9bb6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:27:09.782819 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.782740    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:27:09.782819 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.782762    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6929f886-c638-4507-989e-74f2750c9bb6-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:27:09.782819 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.782777    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:27:09.782819 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.782793    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:27:09.782819 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.782807    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:27:09.784233 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.784196    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6929f886-c638-4507-989e-74f2750c9bb6-istio-podinfo" pod "6929f886-c638-4507-989e-74f2750c9bb6" (UID: "6929f886-c638-4507-989e-74f2750c9bb6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:27:09.784450 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.784414    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-istio-envoy" pod "6929f886-c638-4507-989e-74f2750c9bb6" (UID: "6929f886-c638-4507-989e-74f2750c9bb6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:27:09.784547 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.784421    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6929f886-c638-4507-989e-74f2750c9bb6-istio-token" pod "6929f886-c638-4507-989e-74f2750c9bb6" (UID: "6929f886-c638-4507-989e-74f2750c9bb6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:27:09.784895 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.784866    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6929f886-c638-4507-989e-74f2750c9bb6-kube-api-access-wd2hr" pod "6929f886-c638-4507-989e-74f2750c9bb6" (UID: "6929f886-c638-4507-989e-74f2750c9bb6"). InnerVolumeSpecName "kube-api-access-wd2hr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:27:09.884732 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.884694    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6929f886-c638-4507-989e-74f2750c9bb6-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:27:09.884732 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.884732    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6929f886-c638-4507-989e-74f2750c9bb6-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:27:09.884948 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.884748    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6929f886-c638-4507-989e-74f2750c9bb6-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:27:09.884948 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:09.884762    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wd2hr\" (UniqueName: \"kubernetes.io/projected/6929f886-c638-4507-989e-74f2750c9bb6-kube-api-access-wd2hr\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:27:10.366161 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:10.366122    4942 generic.go:356] "Generic (PLEG): container finished" podID="6929f886-c638-4507-989e-74f2750c9bb6" containerID="f0414cfe9d263bfaacd530f704efd5b0095e1487b440b6b0535b90ba91abd520" exitCode=0
Nov 02 12:27:10.366388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:10.366184    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"
Nov 02 12:27:10.366388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:10.366201    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd" event={"ID":"6929f886-c638-4507-989e-74f2750c9bb6","Type":"ContainerDied","Data":"f0414cfe9d263bfaacd530f704efd5b0095e1487b440b6b0535b90ba91abd520"}
Nov 02 12:27:10.366388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:10.366268    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd" event={"ID":"6929f886-c638-4507-989e-74f2750c9bb6","Type":"ContainerDied","Data":"8ebace5e174e07c674a3870b25dd5a4fff28f7180f6b72d982bf176a2f6cb259"}
Nov 02 12:27:10.366388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:10.366291    4942 scope.go:122] "RemoveContainer" containerID="f0414cfe9d263bfaacd530f704efd5b0095e1487b440b6b0535b90ba91abd520"
Nov 02 12:27:10.380371 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:10.380342    4942 scope.go:122] "RemoveContainer" containerID="515f8297836cc66d087e308006d234fcde215bc09079dc524709e613644da8c6"
Nov 02 12:27:10.390987 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:10.390967    4942 scope.go:122] "RemoveContainer" containerID="f0414cfe9d263bfaacd530f704efd5b0095e1487b440b6b0535b90ba91abd520"
Nov 02 12:27:10.391320 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:27:10.391288    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0414cfe9d263bfaacd530f704efd5b0095e1487b440b6b0535b90ba91abd520\": container with ID starting with f0414cfe9d263bfaacd530f704efd5b0095e1487b440b6b0535b90ba91abd520 not found: ID does not exist" containerID="f0414cfe9d263bfaacd530f704efd5b0095e1487b440b6b0535b90ba91abd520"
Nov 02 12:27:10.391411 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:10.391332    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0414cfe9d263bfaacd530f704efd5b0095e1487b440b6b0535b90ba91abd520"} err="failed to get container status \"f0414cfe9d263bfaacd530f704efd5b0095e1487b440b6b0535b90ba91abd520\": rpc error: code = NotFound desc = could not find container \"f0414cfe9d263bfaacd530f704efd5b0095e1487b440b6b0535b90ba91abd520\": container with ID starting with f0414cfe9d263bfaacd530f704efd5b0095e1487b440b6b0535b90ba91abd520 not found: ID does not exist"
Nov 02 12:27:10.391411 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:10.391358    4942 scope.go:122] "RemoveContainer" containerID="515f8297836cc66d087e308006d234fcde215bc09079dc524709e613644da8c6"
Nov 02 12:27:10.391668 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:27:10.391643    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"515f8297836cc66d087e308006d234fcde215bc09079dc524709e613644da8c6\": container with ID starting with 515f8297836cc66d087e308006d234fcde215bc09079dc524709e613644da8c6 not found: ID does not exist" containerID="515f8297836cc66d087e308006d234fcde215bc09079dc524709e613644da8c6"
Nov 02 12:27:10.391712 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:10.391679    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"515f8297836cc66d087e308006d234fcde215bc09079dc524709e613644da8c6"} err="failed to get container status \"515f8297836cc66d087e308006d234fcde215bc09079dc524709e613644da8c6\": rpc error: code = NotFound desc = could not find container \"515f8297836cc66d087e308006d234fcde215bc09079dc524709e613644da8c6\": container with ID starting with 515f8297836cc66d087e308006d234fcde215bc09079dc524709e613644da8c6 not found: ID does not exist"
Nov 02 12:27:10.585843 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:10.585804    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"]
Nov 02 12:27:10.639496 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:10.639407    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6a432197-openshift-default-6c5bc96b97-f62hd"]
Nov 02 12:27:11.544124 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:11.544086    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6929f886-c638-4507-989e-74f2750c9bb6" path="/var/lib/kubelet/pods/6929f886-c638-4507-989e-74f2750c9bb6/volumes"
Nov 02 12:27:16.837790 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:16.837416    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-62r9f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 12:27:16.837790 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:16.837495    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f" podUID="e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:27:17.334472 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:17.334416    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-jmqgd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 12:27:17.334785 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:17.334737    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd" podUID="d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:27:17.692330 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:17.692264    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-7hsh2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 12:27:17.694077 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:17.694038    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:27:28.431004 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:28.430974    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/ai-tenant-e2e-extmod-b-3a78b4_payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g_e5793ce7-ecac-4c9e-ab91-c5d5723a187f/payload-processing/0.log"
Nov 02 12:27:30.985034 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:27:30.985001    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:27:47.745093 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:47.745054    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"]
Nov 02 12:27:52.746318 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:52.746249    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g" podUID="e5793ce7-ecac-4c9e-ab91-c5d5723a187f" containerName="payload-processing" containerID="cri-o://4d5682393615b890c5eedc80af921907d40085bff5e8c6448123fddb7fbcd834" gracePeriod=25
Nov 02 12:27:53.287606 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.287572    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"
Nov 02 12:27:53.387790 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.387746    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-plugins-config-volume\") pod \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\" (UID: \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\") "
Nov 02 12:27:53.387986 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.387912    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-provider-credentials\" (UniqueName: \"kubernetes.io/projected/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-provider-credentials\") pod \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\" (UID: \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\") "
Nov 02 12:27:53.388059 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.387984    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-routing-overlay-volume\") pod \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\" (UID: \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\") "
Nov 02 12:27:53.388059 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.388033    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-kube-api-access-55c74\" (UniqueName: \"kubernetes.io/projected/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-kube-api-access-55c74\") pod \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\" (UID: \"e5793ce7-ecac-4c9e-ab91-c5d5723a187f\") "
Nov 02 12:27:53.388249 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.388207    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-plugins-config-volume" pod "e5793ce7-ecac-4c9e-ab91-c5d5723a187f" (UID: "e5793ce7-ecac-4c9e-ab91-c5d5723a187f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:27:53.388605 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.388526    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-routing-overlay-volume" pod "e5793ce7-ecac-4c9e-ab91-c5d5723a187f" (UID: "e5793ce7-ecac-4c9e-ab91-c5d5723a187f"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:27:53.389285 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.389261    4942 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-plugins-config-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:27:53.389382 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.389292    4942 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-routing-overlay-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:27:53.391124 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.391097    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-kube-api-access-55c74" pod "e5793ce7-ecac-4c9e-ab91-c5d5723a187f" (UID: "e5793ce7-ecac-4c9e-ab91-c5d5723a187f"). InnerVolumeSpecName "kube-api-access-55c74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:27:53.391355 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.391337    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-provider-credentials" pod "e5793ce7-ecac-4c9e-ab91-c5d5723a187f" (UID: "e5793ce7-ecac-4c9e-ab91-c5d5723a187f"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:27:53.490578 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.490501    4942 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-provider-credentials\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:27:53.490578 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.490536    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55c74\" (UniqueName: \"kubernetes.io/projected/e5793ce7-ecac-4c9e-ab91-c5d5723a187f-kube-api-access-55c74\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:27:53.751424 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.748775    4942 generic.go:356] "Generic (PLEG): container finished" podID="e5793ce7-ecac-4c9e-ab91-c5d5723a187f" containerID="4d5682393615b890c5eedc80af921907d40085bff5e8c6448123fddb7fbcd834" exitCode=0
Nov 02 12:27:53.751424 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.748847    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g" event={"ID":"e5793ce7-ecac-4c9e-ab91-c5d5723a187f","Type":"ContainerDied","Data":"4d5682393615b890c5eedc80af921907d40085bff5e8c6448123fddb7fbcd834"}
Nov 02 12:27:53.751424 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.748904    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g" event={"ID":"e5793ce7-ecac-4c9e-ab91-c5d5723a187f","Type":"ContainerDied","Data":"c8485d599084773b22d23034b1b3e09941e47ed632afdd935023b39dc133fcde"}
Nov 02 12:27:53.751424 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.748938    4942 scope.go:122] "RemoveContainer" containerID="4d5682393615b890c5eedc80af921907d40085bff5e8c6448123fddb7fbcd834"
Nov 02 12:27:53.751424 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.749102    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"
Nov 02 12:27:53.795722 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.795581    4942 scope.go:122] "RemoveContainer" containerID="4d5682393615b890c5eedc80af921907d40085bff5e8c6448123fddb7fbcd834"
Nov 02 12:27:53.796011 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:27:53.795983    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d5682393615b890c5eedc80af921907d40085bff5e8c6448123fddb7fbcd834\": container with ID starting with 4d5682393615b890c5eedc80af921907d40085bff5e8c6448123fddb7fbcd834 not found: ID does not exist" containerID="4d5682393615b890c5eedc80af921907d40085bff5e8c6448123fddb7fbcd834"
Nov 02 12:27:53.796088 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.796026    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d5682393615b890c5eedc80af921907d40085bff5e8c6448123fddb7fbcd834"} err="failed to get container status \"4d5682393615b890c5eedc80af921907d40085bff5e8c6448123fddb7fbcd834\": rpc error: code = NotFound desc = could not find container \"4d5682393615b890c5eedc80af921907d40085bff5e8c6448123fddb7fbcd834\": container with ID starting with 4d5682393615b890c5eedc80af921907d40085bff5e8c6448123fddb7fbcd834 not found: ID does not exist"
Nov 02 12:27:53.918124 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.918080    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"]
Nov 02 12:27:53.968584 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:53.968547    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-3a78b4/payload-processing-external-model-e2e-extmod-b-3a78b4-66b5qgg7g"]
Nov 02 12:27:55.550521 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:27:55.550488    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e5793ce7-ecac-4c9e-ab91-c5d5723a187f" path="/var/lib/kubelet/pods/e5793ce7-ecac-4c9e-ab91-c5d5723a187f/volumes"
Nov 02 12:28:00.983067 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:28:00.983032    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:28:17.097107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:17.097069    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"]
Nov 02 12:28:17.097597 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:17.097344    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w" podUID="6e49a856-d29c-46d8-9606-cc2ba7bc9574" containerName="istio-proxy" containerID="cri-o://2b0f621d86e7489e4a3bb413b2c058717d4e31b189b8460b75a067584fc4399d" gracePeriod=30
Nov 02 12:28:17.146718 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:17.146674    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"]
Nov 02 12:28:17.147007 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:17.146956    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk" podUID="5596075f-9595-476d-a6c2-45234fa26395" containerName="istio-proxy" containerID="cri-o://b07fb56af6453ba38db180a95d7016426ec0afb8df951bc4bc898b481757d18c" gracePeriod=30
Nov 02 12:28:17.150334 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:17.150287    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"]
Nov 02 12:28:17.150611 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:17.150585    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s" podUID="1e181569-4b01-4afe-9d02-278ff66af4ac" containerName="istio-proxy" containerID="cri-o://924d5c2ed796af17cdea7157f2633605f38b383b48650d8a06c023903a0776cc" gracePeriod=30
Nov 02 12:28:17.154428 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:17.154391    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"]
Nov 02 12:28:17.154632 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:17.154585    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s" podUID="1c3ecabc-d682-4784-93fa-765acb5555a4" containerName="istio-proxy" containerID="cri-o://05f523f1951c009e4026f4897c68c42842c7d23f073b8d36bc0e31dacc61df44" gracePeriod=30
Nov 02 12:28:20.856226 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.856192    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"]
Nov 02 12:28:20.857047 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.857027    4942 cpu_manager.go:414] "containerMap: removing container" podUID="6929f886-c638-4507-989e-74f2750c9bb6" containerName="istio-proxy"
Nov 02 12:28:20.857107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.857050    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6929f886-c638-4507-989e-74f2750c9bb6" containerName="istio-proxy"
Nov 02 12:28:20.857107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.857073    4942 cpu_manager.go:414] "containerMap: removing container" podUID="18798180-d3f0-4a87-8226-1b66d45a76bc" containerName="payload-processing"
Nov 02 12:28:20.857107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.857081    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18798180-d3f0-4a87-8226-1b66d45a76bc" containerName="payload-processing"
Nov 02 12:28:20.857107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.857097    4942 cpu_manager.go:414] "containerMap: removing container" podUID="b8373905-a9ab-473e-a698-2e699bb89e33" containerName="istio-proxy"
Nov 02 12:28:20.857107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.857104    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8373905-a9ab-473e-a698-2e699bb89e33" containerName="istio-proxy"
Nov 02 12:28:20.857256 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.857134    4942 cpu_manager.go:414] "containerMap: removing container" podUID="e5793ce7-ecac-4c9e-ab91-c5d5723a187f" containerName="payload-processing"
Nov 02 12:28:20.857256 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.857142    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5793ce7-ecac-4c9e-ab91-c5d5723a187f" containerName="payload-processing"
Nov 02 12:28:20.857256 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.857166    4942 cpu_manager.go:414] "containerMap: removing container" podUID="b632b473-36e0-4dcf-908e-722392bb1a6f" containerName="payload-processing"
Nov 02 12:28:20.857256 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.857173    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b632b473-36e0-4dcf-908e-722392bb1a6f" containerName="payload-processing"
Nov 02 12:28:20.857405 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.857311    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="6929f886-c638-4507-989e-74f2750c9bb6" containerName="istio-proxy"
Nov 02 12:28:20.857405 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.857323    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="18798180-d3f0-4a87-8226-1b66d45a76bc" containerName="payload-processing"
Nov 02 12:28:20.857405 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.857333    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="6929f886-c638-4507-989e-74f2750c9bb6" containerName="istio-proxy"
Nov 02 12:28:20.857405 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.857344    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8373905-a9ab-473e-a698-2e699bb89e33" containerName="istio-proxy"
Nov 02 12:28:20.857405 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.857355    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8373905-a9ab-473e-a698-2e699bb89e33" containerName="istio-proxy"
Nov 02 12:28:20.857405 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.857366    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5793ce7-ecac-4c9e-ab91-c5d5723a187f" containerName="payload-processing"
Nov 02 12:28:20.857405 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.857378    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="b632b473-36e0-4dcf-908e-722392bb1a6f" containerName="payload-processing"
Nov 02 12:28:20.861169 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.861150    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:20.913733 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.913706    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-3a78b4-openshift-default-dockercfg-84mvc\""
Nov 02 12:28:20.941352 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.941293    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-workload-socket\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:20.941478 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.941362    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istiod-ca-cert\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:20.941523 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.941499    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-credential-socket\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:20.941561 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.941535    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-envoy\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:20.941630 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.941614    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-workload-certs\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:20.941666 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.941654    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7dkq\" (UniqueName: \"kubernetes.io/projected/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-kube-api-access-p7dkq\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:20.941699 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.941684    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-data\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:20.941862 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.941850    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-podinfo\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:20.941896 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.941872    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-token\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:20.968230 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:20.968197    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"]
Nov 02 12:28:21.042693 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.042657    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-credential-socket\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.042867 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.042705    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-envoy\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.042867 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.042749    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-workload-certs\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.042867 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.042785    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7dkq\" (UniqueName: \"kubernetes.io/projected/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-kube-api-access-p7dkq\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.042867 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.042823    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-data\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.043088 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.042946    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-podinfo\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.043088 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.042976    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-token\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.043088 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.043072    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-credential-socket\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.043246 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.043123    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-workload-socket\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.043246 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.043138    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-workload-certs\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.043246 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.043219    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istiod-ca-cert\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.043411 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.043265    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-data\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.043471 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.043447    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-workload-socket\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.043884 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.043859    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istiod-ca-cert\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.045017 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.044998    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-envoy\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.045344 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.045323    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-podinfo\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.103859 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.103829    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-token\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.104005 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.103985    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7dkq\" (UniqueName: \"kubernetes.io/projected/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-kube-api-access-p7dkq\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.175215 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.175147    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:21.605216 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.605188    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"]
Nov 02 12:28:21.606552 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:28:21.606521    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4cfa7a7_872f_45d8_aacd_e7fca4cc2e57.slice/crio-27b61a47b58dd87893600b9496131f5687579330a50c4afd78bedb56eca3c5d6 WatchSource:0}: Error finding container 27b61a47b58dd87893600b9496131f5687579330a50c4afd78bedb56eca3c5d6: Status 404 returned error can't find the container with id 27b61a47b58dd87893600b9496131f5687579330a50c4afd78bedb56eca3c5d6
Nov 02 12:28:21.608878 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.608841    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:28:21.608963 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.608906    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:28:21.608963 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:21.608937    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:28:22.009650 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.009566    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m" event={"ID":"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57","Type":"ContainerStarted","Data":"89308f1d9bd71c48fec93559fbc5d56c2c499645572ab713b8d1837cebb02f36"}
Nov 02 12:28:22.009650 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.009625    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m" event={"ID":"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57","Type":"ContainerStarted","Data":"27b61a47b58dd87893600b9496131f5687579330a50c4afd78bedb56eca3c5d6"}
Nov 02 12:28:22.130619 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.130539    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m" podStartSLOduration=2.130522853 podStartE2EDuration="2.130522853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:28:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:28:22.129415202 +0000 UTC m=+1928.700764958" watchObservedRunningTime="2026-11-02 12:28:22.130522853 +0000 UTC m=+1928.701872612"
Nov 02 12:28:22.176177 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.176102    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:22.178199 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.178169    4942 patch_prober.go:29] interesting pod/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 02 12:28:22.178559 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.178532    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m" podUID="a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 02 12:28:22.453195 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.452870    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:28:22.458663 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.458632    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.566319    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-workload-certs\") pod \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") "
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.566386    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-envoy\") pod \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") "
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.566874    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-workload-certs" pod "6e49a856-d29c-46d8-9606-cc2ba7bc9574" (UID: "6e49a856-d29c-46d8-9606-cc2ba7bc9574"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.567242    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-workload-certs\") pod \"5596075f-9595-476d-a6c2-45234fa26395\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") "
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.567287    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-istio-data\") pod \"5596075f-9595-476d-a6c2-45234fa26395\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") "
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.567347    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-istio-envoy\") pod \"5596075f-9595-476d-a6c2-45234fa26395\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") "
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.567385    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-podinfo\") pod \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") "
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.567438    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istiod-ca-cert\") pod \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") "
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.567504    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-workload-socket\") pod \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") "
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.567534    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-credential-socket\") pod \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") "
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.567567    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5596075f-9595-476d-a6c2-45234fa26395-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5596075f-9595-476d-a6c2-45234fa26395-istiod-ca-cert\") pod \"5596075f-9595-476d-a6c2-45234fa26395\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") "
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.567628    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e49a856-d29c-46d8-9606-cc2ba7bc9574-kube-api-access-4rpcj\" (UniqueName: \"kubernetes.io/projected/6e49a856-d29c-46d8-9606-cc2ba7bc9574-kube-api-access-4rpcj\") pod \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") "
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.567733    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-data\") pod \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") "
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.567763    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-credential-socket\") pod \"5596075f-9595-476d-a6c2-45234fa26395\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") "
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.567858    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5596075f-9595-476d-a6c2-45234fa26395-istio-token\" (UniqueName: \"kubernetes.io/projected/5596075f-9595-476d-a6c2-45234fa26395-istio-token\") pod \"5596075f-9595-476d-a6c2-45234fa26395\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") "
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.567902    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-token\" (UniqueName: \"kubernetes.io/projected/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-token\") pod \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\" (UID: \"6e49a856-d29c-46d8-9606-cc2ba7bc9574\") "
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.567940    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5596075f-9595-476d-a6c2-45234fa26395-kube-api-access-lm684\" (UniqueName: \"kubernetes.io/projected/5596075f-9595-476d-a6c2-45234fa26395-kube-api-access-lm684\") pod \"5596075f-9595-476d-a6c2-45234fa26395\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") "
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.567969    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-workload-socket\") pod \"5596075f-9595-476d-a6c2-45234fa26395\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") "
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.568004    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5596075f-9595-476d-a6c2-45234fa26395-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5596075f-9595-476d-a6c2-45234fa26395-istio-podinfo\") pod \"5596075f-9595-476d-a6c2-45234fa26395\" (UID: \"5596075f-9595-476d-a6c2-45234fa26395\") "
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.568762    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.569469    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5596075f-9595-476d-a6c2-45234fa26395-istiod-ca-cert" pod "5596075f-9595-476d-a6c2-45234fa26395" (UID: "5596075f-9595-476d-a6c2-45234fa26395"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.569822    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-workload-certs" pod "5596075f-9595-476d-a6c2-45234fa26395" (UID: "5596075f-9595-476d-a6c2-45234fa26395"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.570014    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-istio-data" pod "5596075f-9595-476d-a6c2-45234fa26395" (UID: "5596075f-9595-476d-a6c2-45234fa26395"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.570214    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-data" pod "6e49a856-d29c-46d8-9606-cc2ba7bc9574" (UID: "6e49a856-d29c-46d8-9606-cc2ba7bc9574"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.570561    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-credential-socket" pod "5596075f-9595-476d-a6c2-45234fa26395" (UID: "5596075f-9595-476d-a6c2-45234fa26395"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.572388 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.570851    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istiod-ca-cert" pod "6e49a856-d29c-46d8-9606-cc2ba7bc9574" (UID: "6e49a856-d29c-46d8-9606-cc2ba7bc9574"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:28:22.610894 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.592151    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-workload-socket" pod "5596075f-9595-476d-a6c2-45234fa26395" (UID: "5596075f-9595-476d-a6c2-45234fa26395"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.610894 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.602730    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-podinfo" pod "6e49a856-d29c-46d8-9606-cc2ba7bc9574" (UID: "6e49a856-d29c-46d8-9606-cc2ba7bc9574"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:28:22.610894 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.609267    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-istio-envoy" pod "5596075f-9595-476d-a6c2-45234fa26395" (UID: "5596075f-9595-476d-a6c2-45234fa26395"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.610894 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.609522    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-workload-socket" pod "6e49a856-d29c-46d8-9606-cc2ba7bc9574" (UID: "6e49a856-d29c-46d8-9606-cc2ba7bc9574"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.610894 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.609696    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-credential-socket" pod "6e49a856-d29c-46d8-9606-cc2ba7bc9574" (UID: "6e49a856-d29c-46d8-9606-cc2ba7bc9574"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.610894 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.610851    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-token" pod "6e49a856-d29c-46d8-9606-cc2ba7bc9574" (UID: "6e49a856-d29c-46d8-9606-cc2ba7bc9574"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:28:22.610894 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.610887    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5596075f-9595-476d-a6c2-45234fa26395-kube-api-access-lm684" pod "5596075f-9595-476d-a6c2-45234fa26395" (UID: "5596075f-9595-476d-a6c2-45234fa26395"). InnerVolumeSpecName "kube-api-access-lm684". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:28:22.611274 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.611173    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5596075f-9595-476d-a6c2-45234fa26395-istio-podinfo" pod "5596075f-9595-476d-a6c2-45234fa26395" (UID: "5596075f-9595-476d-a6c2-45234fa26395"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:28:22.611521 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.611496    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-envoy" pod "6e49a856-d29c-46d8-9606-cc2ba7bc9574" (UID: "6e49a856-d29c-46d8-9606-cc2ba7bc9574"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.615100 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.615072    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5596075f-9595-476d-a6c2-45234fa26395-istio-token" pod "5596075f-9595-476d-a6c2-45234fa26395" (UID: "5596075f-9595-476d-a6c2-45234fa26395"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:28:22.615516 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.615489    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e49a856-d29c-46d8-9606-cc2ba7bc9574-kube-api-access-4rpcj" pod "6e49a856-d29c-46d8-9606-cc2ba7bc9574" (UID: "6e49a856-d29c-46d8-9606-cc2ba7bc9574"). InnerVolumeSpecName "kube-api-access-4rpcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:28:22.622965 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.622936    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:28:22.624980 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.624960    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:28:22.696324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.695335    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.696324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.695369    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.696324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.695385    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.696324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.695400    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.696324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.695414    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.696324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.695428    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.696324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.695444    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.696324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.695458    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.696324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.695473    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5596075f-9595-476d-a6c2-45234fa26395-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.696324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.695487    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rpcj\" (UniqueName: \"kubernetes.io/projected/6e49a856-d29c-46d8-9606-cc2ba7bc9574-kube-api-access-4rpcj\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.696324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.695502    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.696324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.695518    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.696324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.695533    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5596075f-9595-476d-a6c2-45234fa26395-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.696324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.695546    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e49a856-d29c-46d8-9606-cc2ba7bc9574-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.696324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.695560    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lm684\" (UniqueName: \"kubernetes.io/projected/5596075f-9595-476d-a6c2-45234fa26395-kube-api-access-lm684\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.696324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.695574    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5596075f-9595-476d-a6c2-45234fa26395-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.696324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.695588    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5596075f-9595-476d-a6c2-45234fa26395-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.798731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.796951    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-data\") pod \"1c3ecabc-d682-4784-93fa-765acb5555a4\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") "
Nov 02 12:28:22.798731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.797001    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-workload-socket\") pod \"1c3ecabc-d682-4784-93fa-765acb5555a4\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") "
Nov 02 12:28:22.798731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.797068    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-data\") pod \"1e181569-4b01-4afe-9d02-278ff66af4ac\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") "
Nov 02 12:28:22.798731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.797132    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-workload-certs\") pod \"1e181569-4b01-4afe-9d02-278ff66af4ac\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") "
Nov 02 12:28:22.798731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.797165    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-podinfo\") pod \"1c3ecabc-d682-4784-93fa-765acb5555a4\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") "
Nov 02 12:28:22.798731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.797227    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-envoy\") pod \"1c3ecabc-d682-4784-93fa-765acb5555a4\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") "
Nov 02 12:28:22.798731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.797269    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e181569-4b01-4afe-9d02-278ff66af4ac-kube-api-access-fhvcl\" (UniqueName: \"kubernetes.io/projected/1e181569-4b01-4afe-9d02-278ff66af4ac-kube-api-access-fhvcl\") pod \"1e181569-4b01-4afe-9d02-278ff66af4ac\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") "
Nov 02 12:28:22.798731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.797340    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c3ecabc-d682-4784-93fa-765acb5555a4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c3ecabc-d682-4784-93fa-765acb5555a4-istiod-ca-cert\") pod \"1c3ecabc-d682-4784-93fa-765acb5555a4\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") "
Nov 02 12:28:22.798731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.797375    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1e181569-4b01-4afe-9d02-278ff66af4ac-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e181569-4b01-4afe-9d02-278ff66af4ac-istiod-ca-cert\") pod \"1e181569-4b01-4afe-9d02-278ff66af4ac\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") "
Nov 02 12:28:22.798731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.797433    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-token\" (UniqueName: \"kubernetes.io/projected/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-token\") pod \"1e181569-4b01-4afe-9d02-278ff66af4ac\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") "
Nov 02 12:28:22.798731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.797463    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-credential-socket\") pod \"1c3ecabc-d682-4784-93fa-765acb5555a4\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") "
Nov 02 12:28:22.798731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.797513    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-envoy\") pod \"1e181569-4b01-4afe-9d02-278ff66af4ac\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") "
Nov 02 12:28:22.798731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.797543    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c3ecabc-d682-4784-93fa-765acb5555a4-kube-api-access-d6cwk\" (UniqueName: \"kubernetes.io/projected/1c3ecabc-d682-4784-93fa-765acb5555a4-kube-api-access-d6cwk\") pod \"1c3ecabc-d682-4784-93fa-765acb5555a4\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") "
Nov 02 12:28:22.798731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.797581    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-podinfo\") pod \"1e181569-4b01-4afe-9d02-278ff66af4ac\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") "
Nov 02 12:28:22.798731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.797630    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-workload-certs\") pod \"1c3ecabc-d682-4784-93fa-765acb5555a4\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") "
Nov 02 12:28:22.798731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.797682    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-workload-socket\") pod \"1e181569-4b01-4afe-9d02-278ff66af4ac\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") "
Nov 02 12:28:22.798731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.797716    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-credential-socket\") pod \"1e181569-4b01-4afe-9d02-278ff66af4ac\" (UID: \"1e181569-4b01-4afe-9d02-278ff66af4ac\") "
Nov 02 12:28:22.798731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.797758    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-token\" (UniqueName: \"kubernetes.io/projected/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-token\") pod \"1c3ecabc-d682-4784-93fa-765acb5555a4\" (UID: \"1c3ecabc-d682-4784-93fa-765acb5555a4\") "
Nov 02 12:28:22.801542 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.800241    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e181569-4b01-4afe-9d02-278ff66af4ac-istiod-ca-cert" pod "1e181569-4b01-4afe-9d02-278ff66af4ac" (UID: "1e181569-4b01-4afe-9d02-278ff66af4ac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:28:22.801542 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.800548    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-data" pod "1c3ecabc-d682-4784-93fa-765acb5555a4" (UID: "1c3ecabc-d682-4784-93fa-765acb5555a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.801542 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.800761    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-workload-socket" pod "1c3ecabc-d682-4784-93fa-765acb5555a4" (UID: "1c3ecabc-d682-4784-93fa-765acb5555a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.801542 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.801021    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-token" pod "1c3ecabc-d682-4784-93fa-765acb5555a4" (UID: "1c3ecabc-d682-4784-93fa-765acb5555a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:28:22.801542 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.801331    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-data" pod "1e181569-4b01-4afe-9d02-278ff66af4ac" (UID: "1e181569-4b01-4afe-9d02-278ff66af4ac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.802151 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.801890    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-workload-certs" pod "1e181569-4b01-4afe-9d02-278ff66af4ac" (UID: "1e181569-4b01-4afe-9d02-278ff66af4ac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.804550 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.803222    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-workload-certs" pod "1c3ecabc-d682-4784-93fa-765acb5555a4" (UID: "1c3ecabc-d682-4784-93fa-765acb5555a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.804550 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.803425    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-credential-socket" pod "1e181569-4b01-4afe-9d02-278ff66af4ac" (UID: "1e181569-4b01-4afe-9d02-278ff66af4ac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.804550 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.803695    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-workload-socket" pod "1e181569-4b01-4afe-9d02-278ff66af4ac" (UID: "1e181569-4b01-4afe-9d02-278ff66af4ac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.804550 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.804038    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c3ecabc-d682-4784-93fa-765acb5555a4-istiod-ca-cert" pod "1c3ecabc-d682-4784-93fa-765acb5555a4" (UID: "1c3ecabc-d682-4784-93fa-765acb5555a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:28:22.804550 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.804418    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-credential-socket" pod "1c3ecabc-d682-4784-93fa-765acb5555a4" (UID: "1c3ecabc-d682-4784-93fa-765acb5555a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.804550 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.804521    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-token" pod "1e181569-4b01-4afe-9d02-278ff66af4ac" (UID: "1e181569-4b01-4afe-9d02-278ff66af4ac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:28:22.805342 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.805277    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-podinfo" pod "1e181569-4b01-4afe-9d02-278ff66af4ac" (UID: "1e181569-4b01-4afe-9d02-278ff66af4ac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:28:22.805837 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.805806    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c3ecabc-d682-4784-93fa-765acb5555a4-kube-api-access-d6cwk" pod "1c3ecabc-d682-4784-93fa-765acb5555a4" (UID: "1c3ecabc-d682-4784-93fa-765acb5555a4"). InnerVolumeSpecName "kube-api-access-d6cwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:28:22.807004 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.806968    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-envoy" pod "1e181569-4b01-4afe-9d02-278ff66af4ac" (UID: "1e181569-4b01-4afe-9d02-278ff66af4ac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.807756 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.807695    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-podinfo" pod "1c3ecabc-d682-4784-93fa-765acb5555a4" (UID: "1c3ecabc-d682-4784-93fa-765acb5555a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:28:22.807756 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.807716    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e181569-4b01-4afe-9d02-278ff66af4ac-kube-api-access-fhvcl" pod "1e181569-4b01-4afe-9d02-278ff66af4ac" (UID: "1e181569-4b01-4afe-9d02-278ff66af4ac"). InnerVolumeSpecName "kube-api-access-fhvcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:28:22.809408 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.809383    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-envoy" pod "1c3ecabc-d682-4784-93fa-765acb5555a4" (UID: "1c3ecabc-d682-4784-93fa-765acb5555a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:28:22.900463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.900154    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhvcl\" (UniqueName: \"kubernetes.io/projected/1e181569-4b01-4afe-9d02-278ff66af4ac-kube-api-access-fhvcl\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.900463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.900197    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c3ecabc-d682-4784-93fa-765acb5555a4-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.900463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.900215    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e181569-4b01-4afe-9d02-278ff66af4ac-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.900463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.900229    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.900463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.900244    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.900463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.900257    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.900463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.900271    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6cwk\" (UniqueName: \"kubernetes.io/projected/1c3ecabc-d682-4784-93fa-765acb5555a4-kube-api-access-d6cwk\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.900463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.900284    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.900463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.900317    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.900463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.900331    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.900463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.900344    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.900463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.900357    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.900463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.900370    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.900463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.900383    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.900463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.900398    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.900463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.900411    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e181569-4b01-4afe-9d02-278ff66af4ac-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.900463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.900424    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:22.900463 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:22.900437    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c3ecabc-d682-4784-93fa-765acb5555a4-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:28:23.019981 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.019942    4942 generic.go:356] "Generic (PLEG): container finished" podID="1e181569-4b01-4afe-9d02-278ff66af4ac" containerID="924d5c2ed796af17cdea7157f2633605f38b383b48650d8a06c023903a0776cc" exitCode=0
Nov 02 12:28:23.020378 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.020054    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s" event={"ID":"1e181569-4b01-4afe-9d02-278ff66af4ac","Type":"ContainerDied","Data":"924d5c2ed796af17cdea7157f2633605f38b383b48650d8a06c023903a0776cc"}
Nov 02 12:28:23.020378 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.020099    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s" event={"ID":"1e181569-4b01-4afe-9d02-278ff66af4ac","Type":"ContainerDied","Data":"904f9b6f2ec60b446cfb306b0061cdd200e3f029df648ae7c2892a6459e331a2"}
Nov 02 12:28:23.020378 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.020122    4942 scope.go:122] "RemoveContainer" containerID="924d5c2ed796af17cdea7157f2633605f38b383b48650d8a06c023903a0776cc"
Nov 02 12:28:23.020378 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.020243    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"
Nov 02 12:28:23.027575 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.027548    4942 generic.go:356] "Generic (PLEG): container finished" podID="6e49a856-d29c-46d8-9606-cc2ba7bc9574" containerID="2b0f621d86e7489e4a3bb413b2c058717d4e31b189b8460b75a067584fc4399d" exitCode=0
Nov 02 12:28:23.027704 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.027586    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"
Nov 02 12:28:23.027704 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.027631    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w" event={"ID":"6e49a856-d29c-46d8-9606-cc2ba7bc9574","Type":"ContainerDied","Data":"2b0f621d86e7489e4a3bb413b2c058717d4e31b189b8460b75a067584fc4399d"}
Nov 02 12:28:23.027704 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.027683    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w" event={"ID":"6e49a856-d29c-46d8-9606-cc2ba7bc9574","Type":"ContainerDied","Data":"7c08ba753dcde46afc3ff1ac2d56c1e44eb36ff67e6a43ee66e8704943f9530c"}
Nov 02 12:28:23.035213 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.035184    4942 generic.go:356] "Generic (PLEG): container finished" podID="1c3ecabc-d682-4784-93fa-765acb5555a4" containerID="05f523f1951c009e4026f4897c68c42842c7d23f073b8d36bc0e31dacc61df44" exitCode=0
Nov 02 12:28:23.035286 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.035260    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"
Nov 02 12:28:23.035286 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.035262    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s" event={"ID":"1c3ecabc-d682-4784-93fa-765acb5555a4","Type":"ContainerDied","Data":"05f523f1951c009e4026f4897c68c42842c7d23f073b8d36bc0e31dacc61df44"}
Nov 02 12:28:23.035773 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.035351    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s" event={"ID":"1c3ecabc-d682-4784-93fa-765acb5555a4","Type":"ContainerDied","Data":"79b0030e198b02babd004e9b4b8c5672151d2a3551d1ea3feac4e00db6f7fc51"}
Nov 02 12:28:23.037445 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.037423    4942 generic.go:356] "Generic (PLEG): container finished" podID="5596075f-9595-476d-a6c2-45234fa26395" containerID="b07fb56af6453ba38db180a95d7016426ec0afb8df951bc4bc898b481757d18c" exitCode=0
Nov 02 12:28:23.037527 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.037472    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"
Nov 02 12:28:23.037527 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.037504    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk" event={"ID":"5596075f-9595-476d-a6c2-45234fa26395","Type":"ContainerDied","Data":"b07fb56af6453ba38db180a95d7016426ec0afb8df951bc4bc898b481757d18c"}
Nov 02 12:28:23.037622 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.037549    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk" event={"ID":"5596075f-9595-476d-a6c2-45234fa26395","Type":"ContainerDied","Data":"0d1cd65dcf60e58b52f444da2d3c5bd08fcb65efc06ff0a9d4247779f5dc56a1"}
Nov 02 12:28:23.039909 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.039889    4942 scope.go:122] "RemoveContainer" containerID="a25837e3d41728c482673b3118b65c00e2491104ce4b2b7d12e279887b5c47a4"
Nov 02 12:28:23.051138 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.051113    4942 scope.go:122] "RemoveContainer" containerID="924d5c2ed796af17cdea7157f2633605f38b383b48650d8a06c023903a0776cc"
Nov 02 12:28:23.051416 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:28:23.051397    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"924d5c2ed796af17cdea7157f2633605f38b383b48650d8a06c023903a0776cc\": container with ID starting with 924d5c2ed796af17cdea7157f2633605f38b383b48650d8a06c023903a0776cc not found: ID does not exist" containerID="924d5c2ed796af17cdea7157f2633605f38b383b48650d8a06c023903a0776cc"
Nov 02 12:28:23.051488 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.051423    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"924d5c2ed796af17cdea7157f2633605f38b383b48650d8a06c023903a0776cc"} err="failed to get container status \"924d5c2ed796af17cdea7157f2633605f38b383b48650d8a06c023903a0776cc\": rpc error: code = NotFound desc = could not find container \"924d5c2ed796af17cdea7157f2633605f38b383b48650d8a06c023903a0776cc\": container with ID starting with 924d5c2ed796af17cdea7157f2633605f38b383b48650d8a06c023903a0776cc not found: ID does not exist"
Nov 02 12:28:23.051488 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.051440    4942 scope.go:122] "RemoveContainer" containerID="a25837e3d41728c482673b3118b65c00e2491104ce4b2b7d12e279887b5c47a4"
Nov 02 12:28:23.051716 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:28:23.051698    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a25837e3d41728c482673b3118b65c00e2491104ce4b2b7d12e279887b5c47a4\": container with ID starting with a25837e3d41728c482673b3118b65c00e2491104ce4b2b7d12e279887b5c47a4 not found: ID does not exist" containerID="a25837e3d41728c482673b3118b65c00e2491104ce4b2b7d12e279887b5c47a4"
Nov 02 12:28:23.051774 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.051721    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a25837e3d41728c482673b3118b65c00e2491104ce4b2b7d12e279887b5c47a4"} err="failed to get container status \"a25837e3d41728c482673b3118b65c00e2491104ce4b2b7d12e279887b5c47a4\": rpc error: code = NotFound desc = could not find container \"a25837e3d41728c482673b3118b65c00e2491104ce4b2b7d12e279887b5c47a4\": container with ID starting with a25837e3d41728c482673b3118b65c00e2491104ce4b2b7d12e279887b5c47a4 not found: ID does not exist"
Nov 02 12:28:23.051774 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.051737    4942 scope.go:122] "RemoveContainer" containerID="2b0f621d86e7489e4a3bb413b2c058717d4e31b189b8460b75a067584fc4399d"
Nov 02 12:28:23.110994 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.110967    4942 scope.go:122] "RemoveContainer" containerID="2b0f621d86e7489e4a3bb413b2c058717d4e31b189b8460b75a067584fc4399d"
Nov 02 12:28:23.111312 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:28:23.111272    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b0f621d86e7489e4a3bb413b2c058717d4e31b189b8460b75a067584fc4399d\": container with ID starting with 2b0f621d86e7489e4a3bb413b2c058717d4e31b189b8460b75a067584fc4399d not found: ID does not exist" containerID="2b0f621d86e7489e4a3bb413b2c058717d4e31b189b8460b75a067584fc4399d"
Nov 02 12:28:23.111414 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.111323    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b0f621d86e7489e4a3bb413b2c058717d4e31b189b8460b75a067584fc4399d"} err="failed to get container status \"2b0f621d86e7489e4a3bb413b2c058717d4e31b189b8460b75a067584fc4399d\": rpc error: code = NotFound desc = could not find container \"2b0f621d86e7489e4a3bb413b2c058717d4e31b189b8460b75a067584fc4399d\": container with ID starting with 2b0f621d86e7489e4a3bb413b2c058717d4e31b189b8460b75a067584fc4399d not found: ID does not exist"
Nov 02 12:28:23.111414 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.111352    4942 scope.go:122] "RemoveContainer" containerID="05f523f1951c009e4026f4897c68c42842c7d23f073b8d36bc0e31dacc61df44"
Nov 02 12:28:23.125707 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.125619    4942 scope.go:122] "RemoveContainer" containerID="05f523f1951c009e4026f4897c68c42842c7d23f073b8d36bc0e31dacc61df44"
Nov 02 12:28:23.125932 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:28:23.125902    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05f523f1951c009e4026f4897c68c42842c7d23f073b8d36bc0e31dacc61df44\": container with ID starting with 05f523f1951c009e4026f4897c68c42842c7d23f073b8d36bc0e31dacc61df44 not found: ID does not exist" containerID="05f523f1951c009e4026f4897c68c42842c7d23f073b8d36bc0e31dacc61df44"
Nov 02 12:28:23.126016 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.125943    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05f523f1951c009e4026f4897c68c42842c7d23f073b8d36bc0e31dacc61df44"} err="failed to get container status \"05f523f1951c009e4026f4897c68c42842c7d23f073b8d36bc0e31dacc61df44\": rpc error: code = NotFound desc = could not find container \"05f523f1951c009e4026f4897c68c42842c7d23f073b8d36bc0e31dacc61df44\": container with ID starting with 05f523f1951c009e4026f4897c68c42842c7d23f073b8d36bc0e31dacc61df44 not found: ID does not exist"
Nov 02 12:28:23.126016 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.125966    4942 scope.go:122] "RemoveContainer" containerID="b07fb56af6453ba38db180a95d7016426ec0afb8df951bc4bc898b481757d18c"
Nov 02 12:28:23.146318 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.146282    4942 scope.go:122] "RemoveContainer" containerID="b07fb56af6453ba38db180a95d7016426ec0afb8df951bc4bc898b481757d18c"
Nov 02 12:28:23.146601 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:28:23.146571    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b07fb56af6453ba38db180a95d7016426ec0afb8df951bc4bc898b481757d18c\": container with ID starting with b07fb56af6453ba38db180a95d7016426ec0afb8df951bc4bc898b481757d18c not found: ID does not exist" containerID="b07fb56af6453ba38db180a95d7016426ec0afb8df951bc4bc898b481757d18c"
Nov 02 12:28:23.146693 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.146609    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b07fb56af6453ba38db180a95d7016426ec0afb8df951bc4bc898b481757d18c"} err="failed to get container status \"b07fb56af6453ba38db180a95d7016426ec0afb8df951bc4bc898b481757d18c\": rpc error: code = NotFound desc = could not find container \"b07fb56af6453ba38db180a95d7016426ec0afb8df951bc4bc898b481757d18c\": container with ID starting with b07fb56af6453ba38db180a95d7016426ec0afb8df951bc4bc898b481757d18c not found: ID does not exist"
Nov 02 12:28:23.174419 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:28:23.167699    4942 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5596075f_9595_476d_a6c2_45234fa26395.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c3ecabc_d682_4784_93fa_765acb5555a4.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5596075f_9595_476d_a6c2_45234fa26395.slice/crio-0d1cd65dcf60e58b52f444da2d3c5bd08fcb65efc06ff0a9d4247779f5dc56a1\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c3ecabc_d682_4784_93fa_765acb5555a4.slice/crio-79b0030e198b02babd004e9b4b8c5672151d2a3551d1ea3feac4e00db6f7fc51\": RecentStats: unable to find data in memory cache]"
Nov 02 12:28:23.189469 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.189424    4942 patch_prober.go:29] interesting pod/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 02 12:28:23.189663 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.189498    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m" podUID="a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 02 12:28:23.217144 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.217113    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"]
Nov 02 12:28:23.220840 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.220814    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-srf5s"]
Nov 02 12:28:23.440015 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.439933    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"]
Nov 02 12:28:23.444424 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.444398    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-4952w"]
Nov 02 12:28:23.542890 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.542850    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e181569-4b01-4afe-9d02-278ff66af4ac" path="/var/lib/kubelet/pods/1e181569-4b01-4afe-9d02-278ff66af4ac/volumes"
Nov 02 12:28:23.543380 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.543365    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e49a856-d29c-46d8-9606-cc2ba7bc9574" path="/var/lib/kubelet/pods/6e49a856-d29c-46d8-9606-cc2ba7bc9574/volumes"
Nov 02 12:28:23.587321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.587260    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"]
Nov 02 12:28:23.587521 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.587342    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-t2gpk"]
Nov 02 12:28:23.685265 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.685178    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"]
Nov 02 12:28:23.688956 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:23.688927    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b1703c-openshift-default-6755bbfdc7-bf58s"]
Nov 02 12:28:24.184776 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:24.184732    4942 patch_prober.go:29] interesting pod/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 02 12:28:24.185261 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:24.184794    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m" podUID="a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 02 12:28:25.177746 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:25.177694    4942 patch_prober.go:29] interesting pod/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 02 12:28:25.177934 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:25.177762    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m" podUID="a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 02 12:28:25.548029 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:25.547941    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c3ecabc-d682-4784-93fa-765acb5555a4" path="/var/lib/kubelet/pods/1c3ecabc-d682-4784-93fa-765acb5555a4/volumes"
Nov 02 12:28:25.548710 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:25.548686    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5596075f-9595-476d-a6c2-45234fa26395" path="/var/lib/kubelet/pods/5596075f-9595-476d-a6c2-45234fa26395/volumes"
Nov 02 12:28:26.182672 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:26.182635    4942 patch_prober.go:29] interesting pod/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 02 12:28:26.182967 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:26.182931    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m" podUID="a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 02 12:28:27.177363 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:27.175671    4942 patch_prober.go:29] interesting pod/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 02 12:28:27.177363 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:27.175745    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m" podUID="a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 02 12:28:28.176195 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:28.176162    4942 patch_prober.go:29] interesting pod/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 02 12:28:28.176376 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:28.176219    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m" podUID="a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 02 12:28:29.181733 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:29.181697    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:29.182213 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:29.182101    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:29.182836 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:28:29.182816    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:28:30.982731 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:28:30.982699    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:29:00.982179 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:29:00.982142    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:29:19.642717 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:19.642683    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"]
Nov 02 12:29:19.643121 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:19.642890    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm" podUID="7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" containerName="istio-proxy" containerID="cri-o://78807048a36a37184351d0d9060a28c842655c31d55b0275f8afb3c0d1562883" gracePeriod=30
Nov 02 12:29:21.025879 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:21.025838    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 12:29:21.026357 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:21.025908    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm" podUID="7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 12:29:24.947730 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.947704    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:29:24.989836 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.989812    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-token\" (UniqueName: \"kubernetes.io/projected/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-token\") pod \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") "
Nov 02 12:29:24.989994 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.989851    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-workload-socket\") pod \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") "
Nov 02 12:29:24.989994 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.989901    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-credential-socket\") pod \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") "
Nov 02 12:29:24.989994 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.989977    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-kube-api-access-625s4\" (UniqueName: \"kubernetes.io/projected/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-kube-api-access-625s4\") pod \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") "
Nov 02 12:29:24.990189 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.990078    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-data\") pod \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") "
Nov 02 12:29:24.990189 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.990119    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-envoy\") pod \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") "
Nov 02 12:29:24.990189 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.990187    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-workload-certs\") pod \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") "
Nov 02 12:29:24.990372 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.990267    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-podinfo\") pod \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") "
Nov 02 12:29:24.990372 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.990351    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istiod-ca-cert\") pod \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\" (UID: \"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42\") "
Nov 02 12:29:24.990925 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.990659    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-data" pod "7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" (UID: "7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:29:24.990925 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.990668    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-workload-certs" pod "7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" (UID: "7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:29:24.991089 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.990745    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-credential-socket" pod "7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" (UID: "7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:29:24.991089 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.990846    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:29:24.991089 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.991060    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-workload-socket" pod "7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" (UID: "7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:29:24.991396 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.991254    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istiod-ca-cert" pod "7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" (UID: "7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:29:24.993255 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.993216    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-token" pod "7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" (UID: "7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:29:24.993704 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.993681    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-kube-api-access-625s4" pod "7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" (UID: "7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42"). InnerVolumeSpecName "kube-api-access-625s4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:29:24.993796 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.993771    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-podinfo" pod "7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" (UID: "7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:29:24.993923 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:24.993905    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-envoy" pod "7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" (UID: "7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:29:25.092550 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:25.092510    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:29:25.092550 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:25.092550    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:29:25.092731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:25.092566    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:29:25.092731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:25.092578    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:29:25.092731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:25.092590    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:29:25.092731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:25.092602    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:29:25.092731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:25.092614    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:29:25.092731 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:25.092627    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-625s4\" (UniqueName: \"kubernetes.io/projected/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42-kube-api-access-625s4\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:29:25.492277 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:25.492240    4942 generic.go:356] "Generic (PLEG): container finished" podID="7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" containerID="78807048a36a37184351d0d9060a28c842655c31d55b0275f8afb3c0d1562883" exitCode=0
Nov 02 12:29:25.492548 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:25.492328    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm" event={"ID":"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42","Type":"ContainerDied","Data":"78807048a36a37184351d0d9060a28c842655c31d55b0275f8afb3c0d1562883"}
Nov 02 12:29:25.492548 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:25.492349    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"
Nov 02 12:29:25.492548 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:25.492419    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm" event={"ID":"7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42","Type":"ContainerDied","Data":"bc84c9b723648f848eda41585caf642633837eefc23ee8bef65440a2b99b8429"}
Nov 02 12:29:25.492548 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:25.492443    4942 scope.go:122] "RemoveContainer" containerID="78807048a36a37184351d0d9060a28c842655c31d55b0275f8afb3c0d1562883"
Nov 02 12:29:25.503797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:25.503544    4942 scope.go:122] "RemoveContainer" containerID="78807048a36a37184351d0d9060a28c842655c31d55b0275f8afb3c0d1562883"
Nov 02 12:29:25.503915 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:29:25.503880    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78807048a36a37184351d0d9060a28c842655c31d55b0275f8afb3c0d1562883\": container with ID starting with 78807048a36a37184351d0d9060a28c842655c31d55b0275f8afb3c0d1562883 not found: ID does not exist" containerID="78807048a36a37184351d0d9060a28c842655c31d55b0275f8afb3c0d1562883"
Nov 02 12:29:25.504030 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:25.503933    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78807048a36a37184351d0d9060a28c842655c31d55b0275f8afb3c0d1562883"} err="failed to get container status \"78807048a36a37184351d0d9060a28c842655c31d55b0275f8afb3c0d1562883\": rpc error: code = NotFound desc = could not find container \"78807048a36a37184351d0d9060a28c842655c31d55b0275f8afb3c0d1562883\": container with ID starting with 78807048a36a37184351d0d9060a28c842655c31d55b0275f8afb3c0d1562883 not found: ID does not exist"
Nov 02 12:29:25.658918 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:25.658880    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"]
Nov 02 12:29:25.706795 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:25.706760    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-lg4rm"]
Nov 02 12:29:27.542576 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:27.542546    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" path="/var/lib/kubelet/pods/7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42/volumes"
Nov 02 12:29:30.987289 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:29:30.987251    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:29:47.680331 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:47.679401    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-7hsh2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 12:29:47.680331 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:47.679479    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 12:29:56.422977 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.422887    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"]
Nov 02 12:29:56.423488 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423460    4942 cpu_manager.go:414] "containerMap: removing container" podUID="6e49a856-d29c-46d8-9606-cc2ba7bc9574" containerName="istio-proxy"
Nov 02 12:29:56.423488 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423490    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e49a856-d29c-46d8-9606-cc2ba7bc9574" containerName="istio-proxy"
Nov 02 12:29:56.423580 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423510    4942 cpu_manager.go:414] "containerMap: removing container" podUID="7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" containerName="istio-proxy"
Nov 02 12:29:56.423580 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423516    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" containerName="istio-proxy"
Nov 02 12:29:56.423580 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423522    4942 cpu_manager.go:414] "containerMap: removing container" podUID="5596075f-9595-476d-a6c2-45234fa26395" containerName="istio-proxy"
Nov 02 12:29:56.423580 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423527    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5596075f-9595-476d-a6c2-45234fa26395" containerName="istio-proxy"
Nov 02 12:29:56.423580 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423538    4942 cpu_manager.go:414] "containerMap: removing container" podUID="6929f886-c638-4507-989e-74f2750c9bb6" containerName="istio-proxy"
Nov 02 12:29:56.423580 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423542    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6929f886-c638-4507-989e-74f2750c9bb6" containerName="istio-proxy"
Nov 02 12:29:56.423580 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423556    4942 cpu_manager.go:414] "containerMap: removing container" podUID="1e181569-4b01-4afe-9d02-278ff66af4ac" containerName="istio-proxy"
Nov 02 12:29:56.423580 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423560    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e181569-4b01-4afe-9d02-278ff66af4ac" containerName="istio-proxy"
Nov 02 12:29:56.423580 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423566    4942 cpu_manager.go:414] "containerMap: removing container" podUID="b8373905-a9ab-473e-a698-2e699bb89e33" containerName="istio-proxy"
Nov 02 12:29:56.423580 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423570    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8373905-a9ab-473e-a698-2e699bb89e33" containerName="istio-proxy"
Nov 02 12:29:56.423580 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423577    4942 cpu_manager.go:414] "containerMap: removing container" podUID="1c3ecabc-d682-4784-93fa-765acb5555a4" containerName="istio-proxy"
Nov 02 12:29:56.423580 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423581    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c3ecabc-d682-4784-93fa-765acb5555a4" containerName="istio-proxy"
Nov 02 12:29:56.423955 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423684    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e181569-4b01-4afe-9d02-278ff66af4ac" containerName="istio-proxy"
Nov 02 12:29:56.423955 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423692    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e49a856-d29c-46d8-9606-cc2ba7bc9574" containerName="istio-proxy"
Nov 02 12:29:56.423955 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423699    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c2e17ba-c9fa-4b66-bed8-0d4f938cdb42" containerName="istio-proxy"
Nov 02 12:29:56.423955 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423704    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="5596075f-9595-476d-a6c2-45234fa26395" containerName="istio-proxy"
Nov 02 12:29:56.423955 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.423712    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c3ecabc-d682-4784-93fa-765acb5555a4" containerName="istio-proxy"
Nov 02 12:29:56.427017 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.426996    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:29:56.524261 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.524224    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-openshift-service-ca\") pod \"maas-api-e2e-derive-02304760-56f58db9b6-l4hwm\" (UID: \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:29:56.524480 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.524287    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdntv\" (UniqueName: \"kubernetes.io/projected/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-kube-api-access-mdntv\") pod \"maas-api-e2e-derive-02304760-56f58db9b6-l4hwm\" (UID: \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:29:56.524480 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.524438    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-maas-api-tls\") pod \"maas-api-e2e-derive-02304760-56f58db9b6-l4hwm\" (UID: \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:29:56.524583 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.524548    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-metrics-certs\") pod \"maas-api-e2e-derive-02304760-56f58db9b6-l4hwm\" (UID: \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:29:56.531373 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.531349    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-02304760-serving-cert\""
Nov 02 12:29:56.536569 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.536543    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"]
Nov 02 12:29:56.625997 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.625963    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-openshift-service-ca\") pod \"maas-api-e2e-derive-02304760-56f58db9b6-l4hwm\" (UID: \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:29:56.626166 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.626013    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdntv\" (UniqueName: \"kubernetes.io/projected/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-kube-api-access-mdntv\") pod \"maas-api-e2e-derive-02304760-56f58db9b6-l4hwm\" (UID: \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:29:56.626166 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.626053    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-maas-api-tls\") pod \"maas-api-e2e-derive-02304760-56f58db9b6-l4hwm\" (UID: \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:29:56.626166 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.626112    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-metrics-certs\") pod \"maas-api-e2e-derive-02304760-56f58db9b6-l4hwm\" (UID: \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:29:56.626367 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:29:56.626205    4942 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-02304760-serving-cert: secret "maas-api-e2e-derive-02304760-serving-cert" not found
Nov 02 12:29:56.626367 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:29:56.626287    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-maas-api-tls podName:5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f nodeName:}" failed. No retries permitted until 2026-11-02 12:29:57.126266872 +0000 UTC m=+2023.697616621 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-maas-api-tls") pod "maas-api-e2e-derive-02304760-56f58db9b6-l4hwm" (UID: "5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f") : secret "maas-api-e2e-derive-02304760-serving-cert" not found
Nov 02 12:29:56.626821 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.626796    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-openshift-service-ca\") pod \"maas-api-e2e-derive-02304760-56f58db9b6-l4hwm\" (UID: \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:29:56.628395 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.628375    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-metrics-certs\") pod \"maas-api-e2e-derive-02304760-56f58db9b6-l4hwm\" (UID: \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:29:56.687250 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:56.687180    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdntv\" (UniqueName: \"kubernetes.io/projected/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-kube-api-access-mdntv\") pod \"maas-api-e2e-derive-02304760-56f58db9b6-l4hwm\" (UID: \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:29:57.133195 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:57.133162    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-maas-api-tls\") pod \"maas-api-e2e-derive-02304760-56f58db9b6-l4hwm\" (UID: \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:29:57.135628 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:57.135599    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-maas-api-tls\") pod \"maas-api-e2e-derive-02304760-56f58db9b6-l4hwm\" (UID: \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:29:57.340878 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:57.340843    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:29:57.718641 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:57.718560    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"]
Nov 02 12:29:57.735466 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:29:57.735426    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5de7e84a_61d3_4df0_b59f_5ecb4bf56b9f.slice/crio-f7247d08524d755487f962d335a46b148f9852cbc8f297bbec10e7a445190b0a WatchSource:0}: Error finding container f7247d08524d755487f962d335a46b148f9852cbc8f297bbec10e7a445190b0a: Status 404 returned error can't find the container with id f7247d08524d755487f962d335a46b148f9852cbc8f297bbec10e7a445190b0a
Nov 02 12:29:57.736426 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:57.736408    4942 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 12:29:58.722088 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:58.722053    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm" event={"ID":"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f","Type":"ContainerStarted","Data":"f7247d08524d755487f962d335a46b148f9852cbc8f297bbec10e7a445190b0a"}
Nov 02 12:29:59.730909 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:59.730862    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm" event={"ID":"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f","Type":"ContainerStarted","Data":"0fed6bef581a3bfff0f7d1e8af941b90e5264b4ad3d480435f79f1560e19ef16"}
Nov 02 12:29:59.731284 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:59.730934    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:29:59.850190 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:29:59.850143    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm" podStartSLOduration=2.849454605 podStartE2EDuration="3.850127725s" totalImagesPullingTime="1.00067312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:29:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:29:59.848170421 +0000 UTC m=+2026.419520198" watchObservedRunningTime="2026-11-02 12:29:59.850127725 +0000 UTC m=+2026.421477493"
Nov 02 12:30:00.185679 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:00.185647    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5"]
Nov 02 12:30:00.186227 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:00.186212    4942 cpu_manager.go:414] "containerMap: removing container" podUID="1e181569-4b01-4afe-9d02-278ff66af4ac" containerName="istio-proxy"
Nov 02 12:30:00.186281 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:00.186229    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e181569-4b01-4afe-9d02-278ff66af4ac" containerName="istio-proxy"
Nov 02 12:30:00.186358 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:00.186348    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e181569-4b01-4afe-9d02-278ff66af4ac" containerName="istio-proxy"
Nov 02 12:30:00.189842 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:00.189820    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5"
Nov 02 12:30:00.241798 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:00.241767    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nbw2z\""
Nov 02 12:30:00.272481 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:00.272450    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lp42\" (UniqueName: \"kubernetes.io/projected/8e51c038-2866-4e40-a34b-f264eeecfd7c-kube-api-access-6lp42\") pod \"maas-api-key-cleanup-29893710-k8xc5\" (UID: \"8e51c038-2866-4e40-a34b-f264eeecfd7c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5"
Nov 02 12:30:00.272735 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:00.272704    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e51c038-2866-4e40-a34b-f264eeecfd7c-maas-api-service-ca\") pod \"maas-api-key-cleanup-29893710-k8xc5\" (UID: \"8e51c038-2866-4e40-a34b-f264eeecfd7c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5"
Nov 02 12:30:00.278738 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:00.278695    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5"]
Nov 02 12:30:00.373867 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:00.373833    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e51c038-2866-4e40-a34b-f264eeecfd7c-maas-api-service-ca\") pod \"maas-api-key-cleanup-29893710-k8xc5\" (UID: \"8e51c038-2866-4e40-a34b-f264eeecfd7c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5"
Nov 02 12:30:00.374040 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:00.373936    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lp42\" (UniqueName: \"kubernetes.io/projected/8e51c038-2866-4e40-a34b-f264eeecfd7c-kube-api-access-6lp42\") pod \"maas-api-key-cleanup-29893710-k8xc5\" (UID: \"8e51c038-2866-4e40-a34b-f264eeecfd7c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5"
Nov 02 12:30:00.374584 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:00.374562    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e51c038-2866-4e40-a34b-f264eeecfd7c-maas-api-service-ca\") pod \"maas-api-key-cleanup-29893710-k8xc5\" (UID: \"8e51c038-2866-4e40-a34b-f264eeecfd7c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5"
Nov 02 12:30:00.435754 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:00.435678    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lp42\" (UniqueName: \"kubernetes.io/projected/8e51c038-2866-4e40-a34b-f264eeecfd7c-kube-api-access-6lp42\") pod \"maas-api-key-cleanup-29893710-k8xc5\" (UID: \"8e51c038-2866-4e40-a34b-f264eeecfd7c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5"
Nov 02 12:30:00.504736 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:00.504709    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5"
Nov 02 12:30:00.883088 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:00.883063    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5"]
Nov 02 12:30:00.884545 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:30:00.884509    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e51c038_2866_4e40_a34b_f264eeecfd7c.slice/crio-3f0281b0e84846313d924a8fa9e13ad8f8bd964048fb39c1936e044df2f9279d WatchSource:0}: Error finding container 3f0281b0e84846313d924a8fa9e13ad8f8bd964048fb39c1936e044df2f9279d: Status 404 returned error can't find the container with id 3f0281b0e84846313d924a8fa9e13ad8f8bd964048fb39c1936e044df2f9279d
Nov 02 12:30:00.992645 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:30:00.992613    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:30:01.745444 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:01.745405    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5" event={"ID":"8e51c038-2866-4e40-a34b-f264eeecfd7c","Type":"ContainerStarted","Data":"ccfbcb09649b0d2a2ce310c6d6a5bc3c8ef1c53ddcca2c330e5c26c59790ee54"}
Nov 02 12:30:01.745593 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:01.745466    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5" event={"ID":"8e51c038-2866-4e40-a34b-f264eeecfd7c","Type":"ContainerStarted","Data":"3f0281b0e84846313d924a8fa9e13ad8f8bd964048fb39c1936e044df2f9279d"}
Nov 02 12:30:01.863418 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:01.863357    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5" podStartSLOduration=1.863338217 podStartE2EDuration="1.863338217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:30:01.859810072 +0000 UTC m=+2028.431159831" watchObservedRunningTime="2026-11-02 12:30:01.863338217 +0000 UTC m=+2028.434688039"
Nov 02 12:30:02.752826 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:02.752788    4942 generic.go:356] "Generic (PLEG): container finished" podID="8e51c038-2866-4e40-a34b-f264eeecfd7c" containerID="ccfbcb09649b0d2a2ce310c6d6a5bc3c8ef1c53ddcca2c330e5c26c59790ee54" exitCode=60
Nov 02 12:30:02.753356 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:02.752874    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5" event={"ID":"8e51c038-2866-4e40-a34b-f264eeecfd7c","Type":"ContainerDied","Data":"ccfbcb09649b0d2a2ce310c6d6a5bc3c8ef1c53ddcca2c330e5c26c59790ee54"}
Nov 02 12:30:02.753356 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:02.753144    4942 scope.go:122] "RemoveContainer" containerID="ccfbcb09649b0d2a2ce310c6d6a5bc3c8ef1c53ddcca2c330e5c26c59790ee54"
Nov 02 12:30:03.763002 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:03.762969    4942 generic.go:356] "Generic (PLEG): container finished" podID="8e51c038-2866-4e40-a34b-f264eeecfd7c" containerID="206ae497c4684db8e1b6dd7ec0abf002782fc2d70253feaff4b27dd3a5be8826" exitCode=60
Nov 02 12:30:03.763478 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:03.763051    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5" event={"ID":"8e51c038-2866-4e40-a34b-f264eeecfd7c","Type":"ContainerDied","Data":"206ae497c4684db8e1b6dd7ec0abf002782fc2d70253feaff4b27dd3a5be8826"}
Nov 02 12:30:03.763478 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:03.763110    4942 scope.go:122] "RemoveContainer" containerID="ccfbcb09649b0d2a2ce310c6d6a5bc3c8ef1c53ddcca2c330e5c26c59790ee54"
Nov 02 12:30:03.763478 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:03.763382    4942 scope.go:122] "RemoveContainer" containerID="206ae497c4684db8e1b6dd7ec0abf002782fc2d70253feaff4b27dd3a5be8826"
Nov 02 12:30:03.763748 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:30:03.763725    4942 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29893710-k8xc5_odh-ai-gateway-infra(8e51c038-2866-4e40-a34b-f264eeecfd7c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5" podUID="8e51c038-2866-4e40-a34b-f264eeecfd7c"
Nov 02 12:30:04.770956 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:04.770926    4942 scope.go:122] "RemoveContainer" containerID="206ae497c4684db8e1b6dd7ec0abf002782fc2d70253feaff4b27dd3a5be8826"
Nov 02 12:30:04.771434 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:30:04.771254    4942 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29893710-k8xc5_odh-ai-gateway-infra(8e51c038-2866-4e40-a34b-f264eeecfd7c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5" podUID="8e51c038-2866-4e40-a34b-f264eeecfd7c"
Nov 02 12:30:05.743687 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:05.743655    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:30:12.399089 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:12.399054    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52"]
Nov 02 12:30:12.403170 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:12.403152    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52"
Nov 02 12:30:12.460748 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:12.460717    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52"]
Nov 02 12:30:12.505836 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:12.505808    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 12:30:12.518797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:12.518764    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/751b51bc-7be4-4628-a5e3-978e753d65e3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae6808dd-d8x52\" (UID: \"751b51bc-7be4-4628-a5e3-978e753d65e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52"
Nov 02 12:30:12.518952 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:12.518817    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/751b51bc-7be4-4628-a5e3-978e753d65e3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae6808dd-d8x52\" (UID: \"751b51bc-7be4-4628-a5e3-978e753d65e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52"
Nov 02 12:30:12.519022 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:12.518947    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kll2b\" (UniqueName: \"kubernetes.io/projected/751b51bc-7be4-4628-a5e3-978e753d65e3-kube-api-access-kll2b\") pod \"maas-api-revoke-sub-ae6808dd-d8x52\" (UID: \"751b51bc-7be4-4628-a5e3-978e753d65e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52"
Nov 02 12:30:12.620235 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:12.620206    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/751b51bc-7be4-4628-a5e3-978e753d65e3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae6808dd-d8x52\" (UID: \"751b51bc-7be4-4628-a5e3-978e753d65e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52"
Nov 02 12:30:12.620396 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:12.620249    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/751b51bc-7be4-4628-a5e3-978e753d65e3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae6808dd-d8x52\" (UID: \"751b51bc-7be4-4628-a5e3-978e753d65e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52"
Nov 02 12:30:12.620396 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:12.620319    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kll2b\" (UniqueName: \"kubernetes.io/projected/751b51bc-7be4-4628-a5e3-978e753d65e3-kube-api-access-kll2b\") pod \"maas-api-revoke-sub-ae6808dd-d8x52\" (UID: \"751b51bc-7be4-4628-a5e3-978e753d65e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52"
Nov 02 12:30:12.620969 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:12.620940    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/751b51bc-7be4-4628-a5e3-978e753d65e3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae6808dd-d8x52\" (UID: \"751b51bc-7be4-4628-a5e3-978e753d65e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52"
Nov 02 12:30:12.622647 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:12.622624    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/751b51bc-7be4-4628-a5e3-978e753d65e3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae6808dd-d8x52\" (UID: \"751b51bc-7be4-4628-a5e3-978e753d65e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52"
Nov 02 12:30:12.680933 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:12.680869    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kll2b\" (UniqueName: \"kubernetes.io/projected/751b51bc-7be4-4628-a5e3-978e753d65e3-kube-api-access-kll2b\") pod \"maas-api-revoke-sub-ae6808dd-d8x52\" (UID: \"751b51bc-7be4-4628-a5e3-978e753d65e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52"
Nov 02 12:30:12.717848 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:12.717825    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52"
Nov 02 12:30:13.110533 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:13.110506    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52"]
Nov 02 12:30:13.111774 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:30:13.111746    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod751b51bc_7be4_4628_a5e3_978e753d65e3.slice/crio-550104249ba2605f17a0e0034a333289aa89c081f7dbfadf37cb3999cf080928 WatchSource:0}: Error finding container 550104249ba2605f17a0e0034a333289aa89c081f7dbfadf37cb3999cf080928: Status 404 returned error can't find the container with id 550104249ba2605f17a0e0034a333289aa89c081f7dbfadf37cb3999cf080928
Nov 02 12:30:13.833442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:13.833361    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52" event={"ID":"751b51bc-7be4-4628-a5e3-978e753d65e3","Type":"ContainerStarted","Data":"db531b34c21564da6eecd343522aedf9d730246c96d26a8a87192686b7166150"}
Nov 02 12:30:13.833442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:13.833427    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52" event={"ID":"751b51bc-7be4-4628-a5e3-978e753d65e3","Type":"ContainerStarted","Data":"550104249ba2605f17a0e0034a333289aa89c081f7dbfadf37cb3999cf080928"}
Nov 02 12:30:13.950955 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:13.950904    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52" podStartSLOduration=1.950890341 podStartE2EDuration="1.950890341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:30:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:30:13.948186758 +0000 UTC m=+2040.519536523" watchObservedRunningTime="2026-11-02 12:30:13.950890341 +0000 UTC m=+2040.522240122"
Nov 02 12:30:14.843893 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:14.843858    4942 generic.go:356] "Generic (PLEG): container finished" podID="751b51bc-7be4-4628-a5e3-978e753d65e3" containerID="db531b34c21564da6eecd343522aedf9d730246c96d26a8a87192686b7166150" exitCode=0
Nov 02 12:30:14.844329 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:14.843938    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52" event={"ID":"751b51bc-7be4-4628-a5e3-978e753d65e3","Type":"ContainerDied","Data":"db531b34c21564da6eecd343522aedf9d730246c96d26a8a87192686b7166150"}
Nov 02 12:30:16.009707 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:16.009676    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52"
Nov 02 12:30:16.058215 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:16.058187    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/751b51bc-7be4-4628-a5e3-978e753d65e3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/751b51bc-7be4-4628-a5e3-978e753d65e3-maas-api-xks-ca\") pod \"751b51bc-7be4-4628-a5e3-978e753d65e3\" (UID: \"751b51bc-7be4-4628-a5e3-978e753d65e3\") "
Nov 02 12:30:16.058395 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:16.058355    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/751b51bc-7be4-4628-a5e3-978e753d65e3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/751b51bc-7be4-4628-a5e3-978e753d65e3-maas-api-openshift-ca\") pod \"751b51bc-7be4-4628-a5e3-978e753d65e3\" (UID: \"751b51bc-7be4-4628-a5e3-978e753d65e3\") "
Nov 02 12:30:16.058395 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:16.058393    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/751b51bc-7be4-4628-a5e3-978e753d65e3-kube-api-access-kll2b\" (UniqueName: \"kubernetes.io/projected/751b51bc-7be4-4628-a5e3-978e753d65e3-kube-api-access-kll2b\") pod \"751b51bc-7be4-4628-a5e3-978e753d65e3\" (UID: \"751b51bc-7be4-4628-a5e3-978e753d65e3\") "
Nov 02 12:30:16.058765 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:16.058737    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/751b51bc-7be4-4628-a5e3-978e753d65e3-maas-api-openshift-ca" pod "751b51bc-7be4-4628-a5e3-978e753d65e3" (UID: "751b51bc-7be4-4628-a5e3-978e753d65e3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:30:16.060282 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:16.060249    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/751b51bc-7be4-4628-a5e3-978e753d65e3-maas-api-xks-ca" pod "751b51bc-7be4-4628-a5e3-978e753d65e3" (UID: "751b51bc-7be4-4628-a5e3-978e753d65e3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:30:16.060477 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:16.060454    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/751b51bc-7be4-4628-a5e3-978e753d65e3-kube-api-access-kll2b" pod "751b51bc-7be4-4628-a5e3-978e753d65e3" (UID: "751b51bc-7be4-4628-a5e3-978e753d65e3"). InnerVolumeSpecName "kube-api-access-kll2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:30:16.160012 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:16.159926    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/751b51bc-7be4-4628-a5e3-978e753d65e3-maas-api-openshift-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:16.160012 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:16.159960    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kll2b\" (UniqueName: \"kubernetes.io/projected/751b51bc-7be4-4628-a5e3-978e753d65e3-kube-api-access-kll2b\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:16.160012 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:16.159974    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/751b51bc-7be4-4628-a5e3-978e753d65e3-maas-api-xks-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:16.536321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:16.536229    4942 scope.go:122] "RemoveContainer" containerID="206ae497c4684db8e1b6dd7ec0abf002782fc2d70253feaff4b27dd3a5be8826"
Nov 02 12:30:16.859074 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:16.859047    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52"
Nov 02 12:30:16.859219 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:16.859079    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52" event={"ID":"751b51bc-7be4-4628-a5e3-978e753d65e3","Type":"ContainerDied","Data":"550104249ba2605f17a0e0034a333289aa89c081f7dbfadf37cb3999cf080928"}
Nov 02 12:30:16.859219 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:16.859131    4942 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="550104249ba2605f17a0e0034a333289aa89c081f7dbfadf37cb3999cf080928"
Nov 02 12:30:17.868469 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:17.868429    4942 generic.go:356] "Generic (PLEG): container finished" podID="8e51c038-2866-4e40-a34b-f264eeecfd7c" containerID="4f3d5adb446690ebd41cbaaf2f29ef0ea21db58d77db0f8755c632f91fff37eb" exitCode=60
Nov 02 12:30:17.868884 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:17.868486    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5" event={"ID":"8e51c038-2866-4e40-a34b-f264eeecfd7c","Type":"ContainerDied","Data":"4f3d5adb446690ebd41cbaaf2f29ef0ea21db58d77db0f8755c632f91fff37eb"}
Nov 02 12:30:17.868884 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:17.868566    4942 scope.go:122] "RemoveContainer" containerID="206ae497c4684db8e1b6dd7ec0abf002782fc2d70253feaff4b27dd3a5be8826"
Nov 02 12:30:17.869032 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:17.869010    4942 scope.go:122] "RemoveContainer" containerID="4f3d5adb446690ebd41cbaaf2f29ef0ea21db58d77db0f8755c632f91fff37eb"
Nov 02 12:30:17.869402 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:30:17.869367    4942 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29893710-k8xc5_odh-ai-gateway-infra(8e51c038-2866-4e40-a34b-f264eeecfd7c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5" podUID="8e51c038-2866-4e40-a34b-f264eeecfd7c"
Nov 02 12:30:18.995335 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:18.995267    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5"]
Nov 02 12:30:19.150646 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:19.150622    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5"
Nov 02 12:30:19.205719 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:19.205654    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e51c038-2866-4e40-a34b-f264eeecfd7c-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e51c038-2866-4e40-a34b-f264eeecfd7c-maas-api-service-ca\") pod \"8e51c038-2866-4e40-a34b-f264eeecfd7c\" (UID: \"8e51c038-2866-4e40-a34b-f264eeecfd7c\") "
Nov 02 12:30:19.205719 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:19.205718    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e51c038-2866-4e40-a34b-f264eeecfd7c-kube-api-access-6lp42\" (UniqueName: \"kubernetes.io/projected/8e51c038-2866-4e40-a34b-f264eeecfd7c-kube-api-access-6lp42\") pod \"8e51c038-2866-4e40-a34b-f264eeecfd7c\" (UID: \"8e51c038-2866-4e40-a34b-f264eeecfd7c\") "
Nov 02 12:30:19.206114 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:19.206086    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e51c038-2866-4e40-a34b-f264eeecfd7c-maas-api-service-ca" pod "8e51c038-2866-4e40-a34b-f264eeecfd7c" (UID: "8e51c038-2866-4e40-a34b-f264eeecfd7c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:30:19.207706 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:19.207683    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e51c038-2866-4e40-a34b-f264eeecfd7c-kube-api-access-6lp42" pod "8e51c038-2866-4e40-a34b-f264eeecfd7c" (UID: "8e51c038-2866-4e40-a34b-f264eeecfd7c"). InnerVolumeSpecName "kube-api-access-6lp42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:30:19.307837 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:19.307765    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e51c038-2866-4e40-a34b-f264eeecfd7c-maas-api-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:19.307837 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:19.307793    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lp42\" (UniqueName: \"kubernetes.io/projected/8e51c038-2866-4e40-a34b-f264eeecfd7c-kube-api-access-6lp42\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:19.887790 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:19.887758    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5"
Nov 02 12:30:19.887957 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:19.887783    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5" event={"ID":"8e51c038-2866-4e40-a34b-f264eeecfd7c","Type":"ContainerDied","Data":"3f0281b0e84846313d924a8fa9e13ad8f8bd964048fb39c1936e044df2f9279d"}
Nov 02 12:30:19.887957 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:19.887860    4942 scope.go:122] "RemoveContainer" containerID="4f3d5adb446690ebd41cbaaf2f29ef0ea21db58d77db0f8755c632f91fff37eb"
Nov 02 12:30:20.058524 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:20.058489    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5"]
Nov 02 12:30:20.059399 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:20.059373    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29893710-k8xc5"]
Nov 02 12:30:21.545562 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:21.545530    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e51c038-2866-4e40-a34b-f264eeecfd7c" path="/var/lib/kubelet/pods/8e51c038-2866-4e40-a34b-f264eeecfd7c/volumes"
Nov 02 12:30:29.134154 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:29.134116    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"]
Nov 02 12:30:30.986805 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:30:30.986768    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:30:35.241686 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:35.241651    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"]
Nov 02 12:30:35.242239 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:35.241892    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22" podUID="efd23a38-957d-44b6-b345-e038bc4549bb" containerName="istio-proxy" containerID="cri-o://09e46f286e313066d6e39fdcde7fd55679ae58d5a7b06bb7584b2f0440372619" gracePeriod=30
Nov 02 12:30:36.101694 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:36.101645    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 12:30:36.101890 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:36.101726    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22" podUID="efd23a38-957d-44b6-b345-e038bc4549bb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 12:30:40.516058 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.515993    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:30:40.575623 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.575596    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-workload-socket\") pod \"efd23a38-957d-44b6-b345-e038bc4549bb\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") "
Nov 02 12:30:40.575767 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.575695    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-istio-data\") pod \"efd23a38-957d-44b6-b345-e038bc4549bb\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") "
Nov 02 12:30:40.575835 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.575785    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efd23a38-957d-44b6-b345-e038bc4549bb-istio-token\" (UniqueName: \"kubernetes.io/projected/efd23a38-957d-44b6-b345-e038bc4549bb-istio-token\") pod \"efd23a38-957d-44b6-b345-e038bc4549bb\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") "
Nov 02 12:30:40.575835 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.575827    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-credential-socket\") pod \"efd23a38-957d-44b6-b345-e038bc4549bb\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") "
Nov 02 12:30:40.575945 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.575855    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efd23a38-957d-44b6-b345-e038bc4549bb-kube-api-access-bnnfh\" (UniqueName: \"kubernetes.io/projected/efd23a38-957d-44b6-b345-e038bc4549bb-kube-api-access-bnnfh\") pod \"efd23a38-957d-44b6-b345-e038bc4549bb\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") "
Nov 02 12:30:40.575945 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.575878    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-workload-socket" pod "efd23a38-957d-44b6-b345-e038bc4549bb" (UID: "efd23a38-957d-44b6-b345-e038bc4549bb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:30:40.575945 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.575893    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-istio-envoy\") pod \"efd23a38-957d-44b6-b345-e038bc4549bb\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") "
Nov 02 12:30:40.576107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.575985    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/efd23a38-957d-44b6-b345-e038bc4549bb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efd23a38-957d-44b6-b345-e038bc4549bb-istiod-ca-cert\") pod \"efd23a38-957d-44b6-b345-e038bc4549bb\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") "
Nov 02 12:30:40.576107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.575991    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-istio-data" pod "efd23a38-957d-44b6-b345-e038bc4549bb" (UID: "efd23a38-957d-44b6-b345-e038bc4549bb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:30:40.576107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.576044    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/efd23a38-957d-44b6-b345-e038bc4549bb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efd23a38-957d-44b6-b345-e038bc4549bb-istio-podinfo\") pod \"efd23a38-957d-44b6-b345-e038bc4549bb\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") "
Nov 02 12:30:40.576283 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.576112    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-workload-certs\") pod \"efd23a38-957d-44b6-b345-e038bc4549bb\" (UID: \"efd23a38-957d-44b6-b345-e038bc4549bb\") "
Nov 02 12:30:40.576283 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.576121    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-credential-socket" pod "efd23a38-957d-44b6-b345-e038bc4549bb" (UID: "efd23a38-957d-44b6-b345-e038bc4549bb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:30:40.576519 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.576451    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-workload-certs" pod "efd23a38-957d-44b6-b345-e038bc4549bb" (UID: "efd23a38-957d-44b6-b345-e038bc4549bb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:30:40.576519 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.576483    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efd23a38-957d-44b6-b345-e038bc4549bb-istiod-ca-cert" pod "efd23a38-957d-44b6-b345-e038bc4549bb" (UID: "efd23a38-957d-44b6-b345-e038bc4549bb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:30:40.576712 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.576676    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:40.576712 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.576696    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efd23a38-957d-44b6-b345-e038bc4549bb-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:40.576712 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.576710    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:40.576896 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.576723    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:40.576896 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.576736    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:40.578053 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.578031    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efd23a38-957d-44b6-b345-e038bc4549bb-istio-token" pod "efd23a38-957d-44b6-b345-e038bc4549bb" (UID: "efd23a38-957d-44b6-b345-e038bc4549bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:30:40.578163 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.578146    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/efd23a38-957d-44b6-b345-e038bc4549bb-istio-podinfo" pod "efd23a38-957d-44b6-b345-e038bc4549bb" (UID: "efd23a38-957d-44b6-b345-e038bc4549bb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:30:40.578460 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.578438    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-istio-envoy" pod "efd23a38-957d-44b6-b345-e038bc4549bb" (UID: "efd23a38-957d-44b6-b345-e038bc4549bb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:30:40.578517 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.578455    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efd23a38-957d-44b6-b345-e038bc4549bb-kube-api-access-bnnfh" pod "efd23a38-957d-44b6-b345-e038bc4549bb" (UID: "efd23a38-957d-44b6-b345-e038bc4549bb"). InnerVolumeSpecName "kube-api-access-bnnfh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:30:40.677701 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.677668    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efd23a38-957d-44b6-b345-e038bc4549bb-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:40.677701 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.677703    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnnfh\" (UniqueName: \"kubernetes.io/projected/efd23a38-957d-44b6-b345-e038bc4549bb-kube-api-access-bnnfh\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:40.677889 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.677719    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efd23a38-957d-44b6-b345-e038bc4549bb-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:40.677889 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:40.677736    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efd23a38-957d-44b6-b345-e038bc4549bb-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:41.139501 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:41.139466    4942 generic.go:356] "Generic (PLEG): container finished" podID="efd23a38-957d-44b6-b345-e038bc4549bb" containerID="09e46f286e313066d6e39fdcde7fd55679ae58d5a7b06bb7584b2f0440372619" exitCode=0
Nov 02 12:30:41.139669 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:41.139510    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"
Nov 02 12:30:41.139669 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:41.139532    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22" event={"ID":"efd23a38-957d-44b6-b345-e038bc4549bb","Type":"ContainerDied","Data":"09e46f286e313066d6e39fdcde7fd55679ae58d5a7b06bb7584b2f0440372619"}
Nov 02 12:30:41.139669 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:41.139584    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22" event={"ID":"efd23a38-957d-44b6-b345-e038bc4549bb","Type":"ContainerDied","Data":"6bc0ccdf28c1f6cdd39d62de38123849b899d1986d60e93437caf359856fb9c8"}
Nov 02 12:30:41.139669 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:41.139615    4942 scope.go:122] "RemoveContainer" containerID="09e46f286e313066d6e39fdcde7fd55679ae58d5a7b06bb7584b2f0440372619"
Nov 02 12:30:41.152881 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:41.152853    4942 scope.go:122] "RemoveContainer" containerID="09e46f286e313066d6e39fdcde7fd55679ae58d5a7b06bb7584b2f0440372619"
Nov 02 12:30:41.153558 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:30:41.153484    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09e46f286e313066d6e39fdcde7fd55679ae58d5a7b06bb7584b2f0440372619\": container with ID starting with 09e46f286e313066d6e39fdcde7fd55679ae58d5a7b06bb7584b2f0440372619 not found: ID does not exist" containerID="09e46f286e313066d6e39fdcde7fd55679ae58d5a7b06bb7584b2f0440372619"
Nov 02 12:30:41.153558 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:41.153522    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09e46f286e313066d6e39fdcde7fd55679ae58d5a7b06bb7584b2f0440372619"} err="failed to get container status \"09e46f286e313066d6e39fdcde7fd55679ae58d5a7b06bb7584b2f0440372619\": rpc error: code = NotFound desc = could not find container \"09e46f286e313066d6e39fdcde7fd55679ae58d5a7b06bb7584b2f0440372619\": container with ID starting with 09e46f286e313066d6e39fdcde7fd55679ae58d5a7b06bb7584b2f0440372619 not found: ID does not exist"
Nov 02 12:30:41.310759 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:41.310726    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"]
Nov 02 12:30:41.334642 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:41.334612    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-46r22"]
Nov 02 12:30:41.337422 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:41.337397    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"]
Nov 02 12:30:41.543920 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:41.543841    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="efd23a38-957d-44b6-b345-e038bc4549bb" path="/var/lib/kubelet/pods/efd23a38-957d-44b6-b345-e038bc4549bb/volumes"
Nov 02 12:30:44.155541 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.155464    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb" podUID="ca691fd2-312b-49cd-9bdd-4010c8fb10fc" containerName="main" containerID="cri-o://3747016f339933d6b9026fd1d2067dd64d8e62b3a3bd7c14d26e1bff6b66d3c2" gracePeriod=45
Nov 02 12:30:44.292362 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.292328    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb" podUID="ca691fd2-312b-49cd-9bdd-4010c8fb10fc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.87:8000/ready\": dial tcp 10.132.0.87:8000: connect: connection refused"
Nov 02 12:30:44.434830 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.434803    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:30:44.516662 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.516631    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-home\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-home\") pod \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") "
Nov 02 12:30:44.516825 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.516725    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-kube-api-access-hbsxd\" (UniqueName: \"kubernetes.io/projected/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-kube-api-access-hbsxd\") pod \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") "
Nov 02 12:30:44.516825 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.516753    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-model-cache\") pod \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") "
Nov 02 12:30:44.516825 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.516783    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-dshm\") pod \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") "
Nov 02 12:30:44.517023 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.516862    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-tls-certs\" (UniqueName: \"kubernetes.io/secret/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-tls-certs\") pod \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") "
Nov 02 12:30:44.517023 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.516959    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-home" pod "ca691fd2-312b-49cd-9bdd-4010c8fb10fc" (UID: "ca691fd2-312b-49cd-9bdd-4010c8fb10fc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:30:44.517023 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.516979    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-tmp-dir\") pod \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\" (UID: \"ca691fd2-312b-49cd-9bdd-4010c8fb10fc\") "
Nov 02 12:30:44.517023 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.516996    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-model-cache" pod "ca691fd2-312b-49cd-9bdd-4010c8fb10fc" (UID: "ca691fd2-312b-49cd-9bdd-4010c8fb10fc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:30:44.517221 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.517198    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-tmp-dir" pod "ca691fd2-312b-49cd-9bdd-4010c8fb10fc" (UID: "ca691fd2-312b-49cd-9bdd-4010c8fb10fc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:30:44.517360 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.517323    4942 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-tmp-dir\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:44.517360 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.517338    4942 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-home\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:44.517360 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.517346    4942 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-model-cache\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:44.519107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.519079    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-tls-certs" pod "ca691fd2-312b-49cd-9bdd-4010c8fb10fc" (UID: "ca691fd2-312b-49cd-9bdd-4010c8fb10fc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:30:44.519401 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.519374    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-dshm" pod "ca691fd2-312b-49cd-9bdd-4010c8fb10fc" (UID: "ca691fd2-312b-49cd-9bdd-4010c8fb10fc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:30:44.519401 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.519390    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-kube-api-access-hbsxd" pod "ca691fd2-312b-49cd-9bdd-4010c8fb10fc" (UID: "ca691fd2-312b-49cd-9bdd-4010c8fb10fc"). InnerVolumeSpecName "kube-api-access-hbsxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:30:44.618850 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.618581    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbsxd\" (UniqueName: \"kubernetes.io/projected/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-kube-api-access-hbsxd\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:44.618850 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.618611    4942 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-dshm\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:44.618850 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:44.618626    4942 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca691fd2-312b-49cd-9bdd-4010c8fb10fc-tls-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:45.168729 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:45.168696    4942 generic.go:356] "Generic (PLEG): container finished" podID="ca691fd2-312b-49cd-9bdd-4010c8fb10fc" containerID="3747016f339933d6b9026fd1d2067dd64d8e62b3a3bd7c14d26e1bff6b66d3c2" exitCode=0
Nov 02 12:30:45.169238 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:45.168753    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"
Nov 02 12:30:45.169238 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:45.168783    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb" event={"ID":"ca691fd2-312b-49cd-9bdd-4010c8fb10fc","Type":"ContainerDied","Data":"3747016f339933d6b9026fd1d2067dd64d8e62b3a3bd7c14d26e1bff6b66d3c2"}
Nov 02 12:30:45.169238 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:45.168829    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb" event={"ID":"ca691fd2-312b-49cd-9bdd-4010c8fb10fc","Type":"ContainerDied","Data":"49040151e9c4753c842bb5113a7cac6684f0d5221dc193b9e7b671e60744563f"}
Nov 02 12:30:45.169238 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:45.168849    4942 scope.go:122] "RemoveContainer" containerID="3747016f339933d6b9026fd1d2067dd64d8e62b3a3bd7c14d26e1bff6b66d3c2"
Nov 02 12:30:45.179260 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:45.179243    4942 scope.go:122] "RemoveContainer" containerID="3747016f339933d6b9026fd1d2067dd64d8e62b3a3bd7c14d26e1bff6b66d3c2"
Nov 02 12:30:45.179597 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:30:45.179577    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3747016f339933d6b9026fd1d2067dd64d8e62b3a3bd7c14d26e1bff6b66d3c2\": container with ID starting with 3747016f339933d6b9026fd1d2067dd64d8e62b3a3bd7c14d26e1bff6b66d3c2 not found: ID does not exist" containerID="3747016f339933d6b9026fd1d2067dd64d8e62b3a3bd7c14d26e1bff6b66d3c2"
Nov 02 12:30:45.179646 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:45.179605    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3747016f339933d6b9026fd1d2067dd64d8e62b3a3bd7c14d26e1bff6b66d3c2"} err="failed to get container status \"3747016f339933d6b9026fd1d2067dd64d8e62b3a3bd7c14d26e1bff6b66d3c2\": rpc error: code = NotFound desc = could not find container \"3747016f339933d6b9026fd1d2067dd64d8e62b3a3bd7c14d26e1bff6b66d3c2\": container with ID starting with 3747016f339933d6b9026fd1d2067dd64d8e62b3a3bd7c14d26e1bff6b66d3c2 not found: ID does not exist"
Nov 02 12:30:45.338892 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:45.338855    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"]
Nov 02 12:30:45.343543 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:45.343507    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-b1703c/distinct-w0-b1703c-kserve-7ff9784d57-vtchb"]
Nov 02 12:30:45.545831 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:45.545745    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca691fd2-312b-49cd-9bdd-4010c8fb10fc" path="/var/lib/kubelet/pods/ca691fd2-312b-49cd-9bdd-4010c8fb10fc/volumes"
Nov 02 12:30:56.358103 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.358029    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4" podUID="c96d0961-c5a0-4cdc-a6fb-21f0a75329df" containerName="main" containerID="cri-o://4639b5c3298cc9df0cbc4eaa33e3c713f506bd95fd7bb4fee098db22518b1c89" gracePeriod=45
Nov 02 12:30:56.637377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.637350    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:30:56.776061 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.776027    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-kube-api-access-twlsr\" (UniqueName: \"kubernetes.io/projected/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-kube-api-access-twlsr\") pod \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") "
Nov 02 12:30:56.776241 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.776145    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-tmp-dir\") pod \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") "
Nov 02 12:30:56.776241 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.776178    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-model-cache\") pod \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") "
Nov 02 12:30:56.776241 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.776211    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-dshm\") pod \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") "
Nov 02 12:30:56.776466 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.776322    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-home\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-home\") pod \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") "
Nov 02 12:30:56.776466 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.776453    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-tls-certs\" (UniqueName: \"kubernetes.io/secret/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-tls-certs\") pod \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\" (UID: \"c96d0961-c5a0-4cdc-a6fb-21f0a75329df\") "
Nov 02 12:30:56.776583 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.776463    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-model-cache" pod "c96d0961-c5a0-4cdc-a6fb-21f0a75329df" (UID: "c96d0961-c5a0-4cdc-a6fb-21f0a75329df"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:30:56.776924 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.776883    4942 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-model-cache\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:56.776924 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.776911    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-home" pod "c96d0961-c5a0-4cdc-a6fb-21f0a75329df" (UID: "c96d0961-c5a0-4cdc-a6fb-21f0a75329df"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:30:56.777311 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.777270    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-tmp-dir" pod "c96d0961-c5a0-4cdc-a6fb-21f0a75329df" (UID: "c96d0961-c5a0-4cdc-a6fb-21f0a75329df"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:30:56.778819 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.778792    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-kube-api-access-twlsr" pod "c96d0961-c5a0-4cdc-a6fb-21f0a75329df" (UID: "c96d0961-c5a0-4cdc-a6fb-21f0a75329df"). InnerVolumeSpecName "kube-api-access-twlsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:30:56.779198 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.779161    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-tls-certs" pod "c96d0961-c5a0-4cdc-a6fb-21f0a75329df" (UID: "c96d0961-c5a0-4cdc-a6fb-21f0a75329df"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:30:56.779469 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.779438    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-dshm" pod "c96d0961-c5a0-4cdc-a6fb-21f0a75329df" (UID: "c96d0961-c5a0-4cdc-a6fb-21f0a75329df"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:30:56.877951 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.877878    4942 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-home\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:56.877951 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.877905    4942 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-tls-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:56.877951 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.877917    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twlsr\" (UniqueName: \"kubernetes.io/projected/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-kube-api-access-twlsr\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:56.877951 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.877929    4942 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-tmp-dir\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:56.877951 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:56.877937    4942 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c96d0961-c5a0-4cdc-a6fb-21f0a75329df-dshm\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:30:57.267344 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:57.267228    4942 generic.go:356] "Generic (PLEG): container finished" podID="c96d0961-c5a0-4cdc-a6fb-21f0a75329df" containerID="4639b5c3298cc9df0cbc4eaa33e3c713f506bd95fd7bb4fee098db22518b1c89" exitCode=0
Nov 02 12:30:57.267344 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:57.267269    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"
Nov 02 12:30:57.267344 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:57.267328    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4" event={"ID":"c96d0961-c5a0-4cdc-a6fb-21f0a75329df","Type":"ContainerDied","Data":"4639b5c3298cc9df0cbc4eaa33e3c713f506bd95fd7bb4fee098db22518b1c89"}
Nov 02 12:30:57.267568 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:57.267372    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4" event={"ID":"c96d0961-c5a0-4cdc-a6fb-21f0a75329df","Type":"ContainerDied","Data":"c62ddba983bf40511e784a742b452ba8b9b0ccda3a7b3b1ca2f87db32ce9d925"}
Nov 02 12:30:57.267568 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:57.267395    4942 scope.go:122] "RemoveContainer" containerID="4639b5c3298cc9df0cbc4eaa33e3c713f506bd95fd7bb4fee098db22518b1c89"
Nov 02 12:30:57.277706 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:57.277545    4942 scope.go:122] "RemoveContainer" containerID="4639b5c3298cc9df0cbc4eaa33e3c713f506bd95fd7bb4fee098db22518b1c89"
Nov 02 12:30:57.278096 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:30:57.278073    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4639b5c3298cc9df0cbc4eaa33e3c713f506bd95fd7bb4fee098db22518b1c89\": container with ID starting with 4639b5c3298cc9df0cbc4eaa33e3c713f506bd95fd7bb4fee098db22518b1c89 not found: ID does not exist" containerID="4639b5c3298cc9df0cbc4eaa33e3c713f506bd95fd7bb4fee098db22518b1c89"
Nov 02 12:30:57.278199 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:57.278106    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4639b5c3298cc9df0cbc4eaa33e3c713f506bd95fd7bb4fee098db22518b1c89"} err="failed to get container status \"4639b5c3298cc9df0cbc4eaa33e3c713f506bd95fd7bb4fee098db22518b1c89\": rpc error: code = NotFound desc = could not find container \"4639b5c3298cc9df0cbc4eaa33e3c713f506bd95fd7bb4fee098db22518b1c89\": container with ID starting with 4639b5c3298cc9df0cbc4eaa33e3c713f506bd95fd7bb4fee098db22518b1c89 not found: ID does not exist"
Nov 02 12:30:57.434770 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:57.434293    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"]
Nov 02 12:30:57.477579 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:57.477546    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w4-dd087a/sub-test-model-dd087a-kserve-5bd9f85c6f-d55c4"]
Nov 02 12:30:57.543706 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:30:57.543626    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c96d0961-c5a0-4cdc-a6fb-21f0a75329df" path="/var/lib/kubelet/pods/c96d0961-c5a0-4cdc-a6fb-21f0a75329df/volumes"
Nov 02 12:31:00.985909 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:31:00.985872    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:31:13.752579 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:13.752538    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"]
Nov 02 12:31:13.753141 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:13.752784    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm" podUID="5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f" containerName="maas-api" containerID="cri-o://0fed6bef581a3bfff0f7d1e8af941b90e5264b4ad3d480435f79f1560e19ef16" gracePeriod=30
Nov 02 12:31:13.867814 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:13.867784    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:31:13.877221 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:13.877198    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:31:14.050693 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.050669    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:31:14.186636 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.186588    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-openshift-service-ca\") pod \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\" (UID: \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\") "
Nov 02 12:31:14.186810 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.186754    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-kube-api-access-mdntv\" (UniqueName: \"kubernetes.io/projected/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-kube-api-access-mdntv\") pod \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\" (UID: \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\") "
Nov 02 12:31:14.186810 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.186783    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-metrics-certs\") pod \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\" (UID: \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\") "
Nov 02 12:31:14.186901 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.186834    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-maas-api-tls\") pod \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\" (UID: \"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f\") "
Nov 02 12:31:14.187706 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.187640    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-openshift-service-ca" pod "5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f" (UID: "5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:31:14.190048 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.190018    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-metrics-certs" pod "5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f" (UID: "5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:31:14.190180 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.190049    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-maas-api-tls" pod "5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f" (UID: "5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:31:14.190744 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.190719    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-kube-api-access-mdntv" pod "5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f" (UID: "5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f"). InnerVolumeSpecName "kube-api-access-mdntv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:31:14.288606 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.288536    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdntv\" (UniqueName: \"kubernetes.io/projected/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-kube-api-access-mdntv\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:31:14.288606 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.288566    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:31:14.288606 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.288577    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:31:14.288606 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.288586    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:31:14.386074 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.386034    4942 generic.go:356] "Generic (PLEG): container finished" podID="5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f" containerID="0fed6bef581a3bfff0f7d1e8af941b90e5264b4ad3d480435f79f1560e19ef16" exitCode=0
Nov 02 12:31:14.386277 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.386135    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm" event={"ID":"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f","Type":"ContainerDied","Data":"0fed6bef581a3bfff0f7d1e8af941b90e5264b4ad3d480435f79f1560e19ef16"}
Nov 02 12:31:14.386277 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.386136    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"
Nov 02 12:31:14.386277 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.386222    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm" event={"ID":"5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f","Type":"ContainerDied","Data":"f7247d08524d755487f962d335a46b148f9852cbc8f297bbec10e7a445190b0a"}
Nov 02 12:31:14.386277 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.386257    4942 scope.go:122] "RemoveContainer" containerID="0fed6bef581a3bfff0f7d1e8af941b90e5264b4ad3d480435f79f1560e19ef16"
Nov 02 12:31:14.399458 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.399430    4942 scope.go:122] "RemoveContainer" containerID="0fed6bef581a3bfff0f7d1e8af941b90e5264b4ad3d480435f79f1560e19ef16"
Nov 02 12:31:14.399797 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:31:14.399738    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fed6bef581a3bfff0f7d1e8af941b90e5264b4ad3d480435f79f1560e19ef16\": container with ID starting with 0fed6bef581a3bfff0f7d1e8af941b90e5264b4ad3d480435f79f1560e19ef16 not found: ID does not exist" containerID="0fed6bef581a3bfff0f7d1e8af941b90e5264b4ad3d480435f79f1560e19ef16"
Nov 02 12:31:14.399797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.399773    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fed6bef581a3bfff0f7d1e8af941b90e5264b4ad3d480435f79f1560e19ef16"} err="failed to get container status \"0fed6bef581a3bfff0f7d1e8af941b90e5264b4ad3d480435f79f1560e19ef16\": rpc error: code = NotFound desc = could not find container \"0fed6bef581a3bfff0f7d1e8af941b90e5264b4ad3d480435f79f1560e19ef16\": container with ID starting with 0fed6bef581a3bfff0f7d1e8af941b90e5264b4ad3d480435f79f1560e19ef16 not found: ID does not exist"
Nov 02 12:31:14.422319 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:31:14.421848    4942 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5de7e84a_61d3_4df0_b59f_5ecb4bf56b9f.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5de7e84a_61d3_4df0_b59f_5ecb4bf56b9f.slice/crio-f7247d08524d755487f962d335a46b148f9852cbc8f297bbec10e7a445190b0a\": RecentStats: unable to find data in memory cache]"
Nov 02 12:31:14.428979 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:31:14.422860    4942 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5de7e84a_61d3_4df0_b59f_5ecb4bf56b9f.slice/crio-f7247d08524d755487f962d335a46b148f9852cbc8f297bbec10e7a445190b0a\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5de7e84a_61d3_4df0_b59f_5ecb4bf56b9f.slice\": RecentStats: unable to find data in memory cache]"
Nov 02 12:31:14.552763 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.552677    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"]
Nov 02 12:31:14.556040 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:14.556015    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-02304760-56f58db9b6-l4hwm"]
Nov 02 12:31:15.544438 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:31:15.544406    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f" path="/var/lib/kubelet/pods/5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f/volumes"
Nov 02 12:31:30.983280 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:31:30.983198    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:32:00.984019 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:32:00.983982    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:32:30.982251 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:32:30.982213    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:32:32.231570 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.231536    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"]
Nov 02 12:32:32.232089 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.231817    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d" podUID="f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b" containerName="discovery" containerID="cri-o://6bbbbe6434aef334315757c07df8bd27c0cdd17299af80f6e6c0c2b5aaaf3632" gracePeriod=30
Nov 02 12:32:32.489464 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.489398    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:32:32.642896 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.642864    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-local-certs\") pod \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") "
Nov 02 12:32:32.643074 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.642921    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-kubeconfig\") pod \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") "
Nov 02 12:32:32.643074 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.643011    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-kube-api-access-hlxmq\" (UniqueName: \"kubernetes.io/projected/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-kube-api-access-hlxmq\") pod \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") "
Nov 02 12:32:32.643193 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.643088    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-csr-ca-configmap\") pod \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") "
Nov 02 12:32:32.643193 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.643125    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-csr-dns-cert\") pod \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") "
Nov 02 12:32:32.643193 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.643174    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-token\" (UniqueName: \"kubernetes.io/projected/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-token\") pod \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") "
Nov 02 12:32:32.643370 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.643201    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-cacerts\" (UniqueName: \"kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-cacerts\") pod \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\" (UID: \"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b\") "
Nov 02 12:32:32.643627 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.643597    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-csr-ca-configmap" pod "f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b" (UID: "f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:32:32.646472 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.646438    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-csr-dns-cert" pod "f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b" (UID: "f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:32:32.646656 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.646629    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-kubeconfig" pod "f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b" (UID: "f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:32:32.646831 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.646808    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-kube-api-access-hlxmq" pod "f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b" (UID: "f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b"). InnerVolumeSpecName "kube-api-access-hlxmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:32:32.647403 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.647182    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-local-certs" pod "f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b" (UID: "f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:32:32.647403 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.647275    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-cacerts" pod "f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b" (UID: "f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:32:32.648250 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.648223    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-token" pod "f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b" (UID: "f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:32:32.744925 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.744827    4942 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-local-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:32:32.744925 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.744877    4942 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-kubeconfig\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:32:32.744925 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.744888    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlxmq\" (UniqueName: \"kubernetes.io/projected/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-kube-api-access-hlxmq\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:32:32.744925 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.744899    4942 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-csr-ca-configmap\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:32:32.744925 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.744908    4942 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-csr-dns-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:32:32.744925 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.744916    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:32:32.744925 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.744924    4942 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b-cacerts\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:32:32.880960 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.880921    4942 generic.go:356] "Generic (PLEG): container finished" podID="f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b" containerID="6bbbbe6434aef334315757c07df8bd27c0cdd17299af80f6e6c0c2b5aaaf3632" exitCode=0
Nov 02 12:32:32.881151 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.880987    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"
Nov 02 12:32:32.881151 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.881007    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d" event={"ID":"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b","Type":"ContainerDied","Data":"6bbbbe6434aef334315757c07df8bd27c0cdd17299af80f6e6c0c2b5aaaf3632"}
Nov 02 12:32:32.881151 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.881095    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d" event={"ID":"f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b","Type":"ContainerDied","Data":"deca0f20da734fadc8e5bdc8aa63244b075c5c3c9d6ddbe495d8eb5b8e6ecf04"}
Nov 02 12:32:32.881151 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.881124    4942 scope.go:122] "RemoveContainer" containerID="6bbbbe6434aef334315757c07df8bd27c0cdd17299af80f6e6c0c2b5aaaf3632"
Nov 02 12:32:32.892386 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.892364    4942 scope.go:122] "RemoveContainer" containerID="6bbbbe6434aef334315757c07df8bd27c0cdd17299af80f6e6c0c2b5aaaf3632"
Nov 02 12:32:32.892632 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:32:32.892611    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bbbbe6434aef334315757c07df8bd27c0cdd17299af80f6e6c0c2b5aaaf3632\": container with ID starting with 6bbbbe6434aef334315757c07df8bd27c0cdd17299af80f6e6c0c2b5aaaf3632 not found: ID does not exist" containerID="6bbbbe6434aef334315757c07df8bd27c0cdd17299af80f6e6c0c2b5aaaf3632"
Nov 02 12:32:32.892676 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:32.892641    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bbbbe6434aef334315757c07df8bd27c0cdd17299af80f6e6c0c2b5aaaf3632"} err="failed to get container status \"6bbbbe6434aef334315757c07df8bd27c0cdd17299af80f6e6c0c2b5aaaf3632\": rpc error: code = NotFound desc = could not find container \"6bbbbe6434aef334315757c07df8bd27c0cdd17299af80f6e6c0c2b5aaaf3632\": container with ID starting with 6bbbbe6434aef334315757c07df8bd27c0cdd17299af80f6e6c0c2b5aaaf3632 not found: ID does not exist"
Nov 02 12:32:33.046418 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:33.046338    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"]
Nov 02 12:32:33.096755 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:33.096721    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r577d"]
Nov 02 12:32:33.543121 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:33.543087    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b" path="/var/lib/kubelet/pods/f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b/volumes"
Nov 02 12:32:45.019082 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019025    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"]
Nov 02 12:32:45.019646 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019626    4942 cpu_manager.go:414] "containerMap: removing container" podUID="8e51c038-2866-4e40-a34b-f264eeecfd7c" containerName="cleanup"
Nov 02 12:32:45.019696 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019647    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e51c038-2866-4e40-a34b-f264eeecfd7c" containerName="cleanup"
Nov 02 12:32:45.019696 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019660    4942 cpu_manager.go:414] "containerMap: removing container" podUID="5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f" containerName="maas-api"
Nov 02 12:32:45.019696 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019665    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f" containerName="maas-api"
Nov 02 12:32:45.019696 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019675    4942 cpu_manager.go:414] "containerMap: removing container" podUID="8e51c038-2866-4e40-a34b-f264eeecfd7c" containerName="cleanup"
Nov 02 12:32:45.019696 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019681    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e51c038-2866-4e40-a34b-f264eeecfd7c" containerName="cleanup"
Nov 02 12:32:45.019696 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019695    4942 cpu_manager.go:414] "containerMap: removing container" podUID="c96d0961-c5a0-4cdc-a6fb-21f0a75329df" containerName="main"
Nov 02 12:32:45.019873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019700    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c96d0961-c5a0-4cdc-a6fb-21f0a75329df" containerName="main"
Nov 02 12:32:45.019873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019717    4942 cpu_manager.go:414] "containerMap: removing container" podUID="ca691fd2-312b-49cd-9bdd-4010c8fb10fc" containerName="main"
Nov 02 12:32:45.019873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019722    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca691fd2-312b-49cd-9bdd-4010c8fb10fc" containerName="main"
Nov 02 12:32:45.019873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019730    4942 cpu_manager.go:414] "containerMap: removing container" podUID="751b51bc-7be4-4628-a5e3-978e753d65e3" containerName="revoke-keys"
Nov 02 12:32:45.019873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019734    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="751b51bc-7be4-4628-a5e3-978e753d65e3" containerName="revoke-keys"
Nov 02 12:32:45.019873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019741    4942 cpu_manager.go:414] "containerMap: removing container" podUID="f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b" containerName="discovery"
Nov 02 12:32:45.019873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019747    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b" containerName="discovery"
Nov 02 12:32:45.019873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019765    4942 cpu_manager.go:414] "containerMap: removing container" podUID="efd23a38-957d-44b6-b345-e038bc4549bb" containerName="istio-proxy"
Nov 02 12:32:45.019873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019769    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efd23a38-957d-44b6-b345-e038bc4549bb" containerName="istio-proxy"
Nov 02 12:32:45.019873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019841    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca691fd2-312b-49cd-9bdd-4010c8fb10fc" containerName="main"
Nov 02 12:32:45.019873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019850    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="751b51bc-7be4-4628-a5e3-978e753d65e3" containerName="revoke-keys"
Nov 02 12:32:45.019873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019856    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4f4b8c8-2ce1-4d29-88f8-d36848c4e25b" containerName="discovery"
Nov 02 12:32:45.019873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019862    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e51c038-2866-4e40-a34b-f264eeecfd7c" containerName="cleanup"
Nov 02 12:32:45.019873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019868    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="efd23a38-957d-44b6-b345-e038bc4549bb" containerName="istio-proxy"
Nov 02 12:32:45.019873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019874    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e51c038-2866-4e40-a34b-f264eeecfd7c" containerName="cleanup"
Nov 02 12:32:45.020283 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019880    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="5de7e84a-61d3-4df0-b59f-5ecb4bf56b9f" containerName="maas-api"
Nov 02 12:32:45.020283 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019886    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e51c038-2866-4e40-a34b-f264eeecfd7c" containerName="cleanup"
Nov 02 12:32:45.020283 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.019892    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="c96d0961-c5a0-4cdc-a6fb-21f0a75329df" containerName="main"
Nov 02 12:32:45.023196 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.023177    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.091621 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.091581    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-257e6d-openshift-default-dockercfg-shf6k\""
Nov 02 12:32:45.145552 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.145516    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"]
Nov 02 12:32:45.172142 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.172107    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqpl6\" (UniqueName: \"kubernetes.io/projected/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-kube-api-access-bqpl6\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.172431 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.172173    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-workload-socket\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.172431 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.172369    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istiod-ca-cert\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.172583 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.172470    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-podinfo\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.172583 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.172490    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-data\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.172583 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.172516    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-envoy\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.172686 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.172609    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-token\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.172686 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.172673    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-workload-certs\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.172780 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.172751    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-credential-socket\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.273809 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.273720    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-envoy\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.273809 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.273764    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-token\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.274047 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.273825    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-workload-certs\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.274047 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.273898    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-credential-socket\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.274047 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.274031    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqpl6\" (UniqueName: \"kubernetes.io/projected/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-kube-api-access-bqpl6\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.274274 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.274246    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-workload-socket\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.274437 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.274275    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-workload-certs\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.274437 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.274366    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istiod-ca-cert\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.274437 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.274383    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-credential-socket\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.274626 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.274451    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-podinfo\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.274626 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.274479    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-data\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.274626 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.274575    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-workload-socket\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.274828 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.274807    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-data\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.274936 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.274920    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istiod-ca-cert\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.276233 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.276212    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-envoy\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.276727 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.276708    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-podinfo\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.342996 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.342968    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-token\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.344541 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.344521    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqpl6\" (UniqueName: \"kubernetes.io/projected/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-kube-api-access-bqpl6\") pod \"e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") " pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:45.636175 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:45.636147    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:46.065930 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:46.065903    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"]
Nov 02 12:32:46.066256 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:32:46.066054    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c0ef0fa_78a0_414b_944d_a81c3f3f9e3c.slice/crio-e196fedc56560585c0bde2161fa5ae270a3b31eab02e38aa0977268f2b982033 WatchSource:0}: Error finding container e196fedc56560585c0bde2161fa5ae270a3b31eab02e38aa0977268f2b982033: Status 404 returned error can't find the container with id e196fedc56560585c0bde2161fa5ae270a3b31eab02e38aa0977268f2b982033
Nov 02 12:32:46.068359 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:46.068328    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:32:46.068446 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:46.068392    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:32:46.068446 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:46.068422    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:32:46.965638 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:46.965604    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4" event={"ID":"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c","Type":"ContainerStarted","Data":"b6cb99aa29c097599373493eb0427d013e05c0bc8d1b672387298edb6f716407"}
Nov 02 12:32:46.965806 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:46.965655    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4" event={"ID":"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c","Type":"ContainerStarted","Data":"e196fedc56560585c0bde2161fa5ae270a3b31eab02e38aa0977268f2b982033"}
Nov 02 12:32:47.086212 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:47.086160    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4" podStartSLOduration=3.086147211 podStartE2EDuration="3.086147211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:32:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:32:47.085057912 +0000 UTC m=+2193.656407707" watchObservedRunningTime="2026-11-02 12:32:47.086147211 +0000 UTC m=+2193.657496968"
Nov 02 12:32:47.637081 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:47.637049    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:47.642954 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:47.642926    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:47.973155 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:47.973078    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:47.974113 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:47.974094    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:32:53.553595 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:53.553563    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4"]
Nov 02 12:32:53.554077 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:53.554062    4942 cpu_manager.go:414] "containerMap: removing container" podUID="8e51c038-2866-4e40-a34b-f264eeecfd7c" containerName="cleanup"
Nov 02 12:32:53.554120 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:53.554077    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e51c038-2866-4e40-a34b-f264eeecfd7c" containerName="cleanup"
Nov 02 12:32:53.557662 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:53.557638    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4"
Nov 02 12:32:53.660355 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:53.660117    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-257e6d-dockercfg-s579g\""
Nov 02 12:32:53.660355 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:53.660114    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-257e6d\""
Nov 02 12:32:53.663312 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:53.663266    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4"]
Nov 02 12:32:53.676412 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:53.673876    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwbfl\" (UniqueName: \"kubernetes.io/projected/d61ef5ad-d084-49a0-8241-eb808e8df109-kube-api-access-bwbfl\") pod \"payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4\" (UID: \"d61ef5ad-d084-49a0-8241-eb808e8df109\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4"
Nov 02 12:32:53.676412 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:53.674102    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d61ef5ad-d084-49a0-8241-eb808e8df109-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4\" (UID: \"d61ef5ad-d084-49a0-8241-eb808e8df109\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4"
Nov 02 12:32:53.775608 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:53.775575    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwbfl\" (UniqueName: \"kubernetes.io/projected/d61ef5ad-d084-49a0-8241-eb808e8df109-kube-api-access-bwbfl\") pod \"payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4\" (UID: \"d61ef5ad-d084-49a0-8241-eb808e8df109\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4"
Nov 02 12:32:53.775808 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:53.775679    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d61ef5ad-d084-49a0-8241-eb808e8df109-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4\" (UID: \"d61ef5ad-d084-49a0-8241-eb808e8df109\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4"
Nov 02 12:32:53.776393 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:53.776367    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d61ef5ad-d084-49a0-8241-eb808e8df109-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4\" (UID: \"d61ef5ad-d084-49a0-8241-eb808e8df109\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4"
Nov 02 12:32:53.836656 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:53.836590    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwbfl\" (UniqueName: \"kubernetes.io/projected/d61ef5ad-d084-49a0-8241-eb808e8df109-kube-api-access-bwbfl\") pod \"payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4\" (UID: \"d61ef5ad-d084-49a0-8241-eb808e8df109\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4"
Nov 02 12:32:53.872235 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:53.872168    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4"
Nov 02 12:32:54.249368 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:54.249337    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4"]
Nov 02 12:32:54.250178 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:32:54.250152    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd61ef5ad_d084_49a0_8241_eb808e8df109.slice/crio-d7d899814dd683c2910a9abea5d9f615cf21b76b41f2cd415c8c935c86417565 WatchSource:0}: Error finding container d7d899814dd683c2910a9abea5d9f615cf21b76b41f2cd415c8c935c86417565: Status 404 returned error can't find the container with id d7d899814dd683c2910a9abea5d9f615cf21b76b41f2cd415c8c935c86417565
Nov 02 12:32:55.021220 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:55.021188    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4" event={"ID":"d61ef5ad-d084-49a0-8241-eb808e8df109","Type":"ContainerStarted","Data":"a7d15820d1ec158eeef7b1c81d8980228f4f320777e316b21f131fd122efbcbd"}
Nov 02 12:32:55.021641 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:55.021238    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4" event={"ID":"d61ef5ad-d084-49a0-8241-eb808e8df109","Type":"ContainerStarted","Data":"d7d899814dd683c2910a9abea5d9f615cf21b76b41f2cd415c8c935c86417565"}
Nov 02 12:32:55.021641 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:55.021328    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4"
Nov 02 12:32:55.137095 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:32:55.137049    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4" podStartSLOduration=2.137036225 podStartE2EDuration="2.137036225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:32:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:32:55.136979201 +0000 UTC m=+2201.708328958" watchObservedRunningTime="2026-11-02 12:32:55.137036225 +0000 UTC m=+2201.708385982"
Nov 02 12:33:00.984686 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:33:00.984653    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:33:01.309693 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:01.309600    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"]
Nov 02 12:33:06.030374 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:06.030345    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4"
Nov 02 12:33:16.329555 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.329499    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f" podUID="5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" containerName="main" containerID="cri-o://d2cc63bd719bd45f72c078595901861432113fd272e2339f8112e595c4efc9bc" gracePeriod=45
Nov 02 12:33:16.591933 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.591865    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:33:16.734928 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.734895    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-model-cache\") pod \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") "
Nov 02 12:33:16.735097 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.734951    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-tmp-dir\") pod \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") "
Nov 02 12:33:16.735097 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.734999    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-tls-certs\" (UniqueName: \"kubernetes.io/secret/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-tls-certs\") pod \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") "
Nov 02 12:33:16.735097 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.735035    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-kube-api-access-nwk29\" (UniqueName: \"kubernetes.io/projected/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-kube-api-access-nwk29\") pod \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") "
Nov 02 12:33:16.735282 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.735184    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-dshm\") pod \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") "
Nov 02 12:33:16.735282 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.735246    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-home\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-home\") pod \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\" (UID: \"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212\") "
Nov 02 12:33:16.735282 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.735261    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-model-cache" pod "5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" (UID: "5dfe9fdf-8c30-4c80-b3e1-76bf844ae212"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:33:16.735457 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.735277    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-tmp-dir" pod "5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" (UID: "5dfe9fdf-8c30-4c80-b3e1-76bf844ae212"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:33:16.735599 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.735575    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-home" pod "5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" (UID: "5dfe9fdf-8c30-4c80-b3e1-76bf844ae212"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:33:16.735781 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.735761    4942 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-model-cache\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:33:16.735846 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.735786    4942 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-tmp-dir\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:33:16.735846 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.735800    4942 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-home\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:33:16.737433 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.737397    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-kube-api-access-nwk29" pod "5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" (UID: "5dfe9fdf-8c30-4c80-b3e1-76bf844ae212"). InnerVolumeSpecName "kube-api-access-nwk29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:33:16.737572 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.737447    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-tls-certs" pod "5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" (UID: "5dfe9fdf-8c30-4c80-b3e1-76bf844ae212"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:33:16.737658 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.737637    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-dshm" pod "5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" (UID: "5dfe9fdf-8c30-4c80-b3e1-76bf844ae212"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:33:16.837258 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.837221    4942 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-tls-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:33:16.837258 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.837251    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwk29\" (UniqueName: \"kubernetes.io/projected/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-kube-api-access-nwk29\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:33:16.837258 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:16.837262    4942 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212-dshm\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:33:17.160959 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:17.160918    4942 generic.go:356] "Generic (PLEG): container finished" podID="5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" containerID="d2cc63bd719bd45f72c078595901861432113fd272e2339f8112e595c4efc9bc" exitCode=0
Nov 02 12:33:17.161164 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:17.160969    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"
Nov 02 12:33:17.161164 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:17.161009    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f" event={"ID":"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212","Type":"ContainerDied","Data":"d2cc63bd719bd45f72c078595901861432113fd272e2339f8112e595c4efc9bc"}
Nov 02 12:33:17.161164 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:17.161064    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f" event={"ID":"5dfe9fdf-8c30-4c80-b3e1-76bf844ae212","Type":"ContainerDied","Data":"59023a1b1fcc82fdd6d283e214f5b100a070f82e3abeb0267360d0bed8d88c59"}
Nov 02 12:33:17.161164 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:17.161087    4942 scope.go:122] "RemoveContainer" containerID="d2cc63bd719bd45f72c078595901861432113fd272e2339f8112e595c4efc9bc"
Nov 02 12:33:17.171531 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:17.171513    4942 scope.go:122] "RemoveContainer" containerID="d2cc63bd719bd45f72c078595901861432113fd272e2339f8112e595c4efc9bc"
Nov 02 12:33:17.171798 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:33:17.171777    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2cc63bd719bd45f72c078595901861432113fd272e2339f8112e595c4efc9bc\": container with ID starting with d2cc63bd719bd45f72c078595901861432113fd272e2339f8112e595c4efc9bc not found: ID does not exist" containerID="d2cc63bd719bd45f72c078595901861432113fd272e2339f8112e595c4efc9bc"
Nov 02 12:33:17.171835 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:17.171807    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2cc63bd719bd45f72c078595901861432113fd272e2339f8112e595c4efc9bc"} err="failed to get container status \"d2cc63bd719bd45f72c078595901861432113fd272e2339f8112e595c4efc9bc\": rpc error: code = NotFound desc = could not find container \"d2cc63bd719bd45f72c078595901861432113fd272e2339f8112e595c4efc9bc\": container with ID starting with d2cc63bd719bd45f72c078595901861432113fd272e2339f8112e595c4efc9bc not found: ID does not exist"
Nov 02 12:33:17.324617 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:17.324585    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"]
Nov 02 12:33:17.373672 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:17.373645    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-4b9210/simulator-w3-4b9210-kserve-5548b779d-wqg5f"]
Nov 02 12:33:17.543260 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:17.543189    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" path="/var/lib/kubelet/pods/5dfe9fdf-8c30-4c80-b3e1-76bf844ae212/volumes"
Nov 02 12:33:26.264644 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.264610    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"]
Nov 02 12:33:26.265178 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.265162    4942 cpu_manager.go:414] "containerMap: removing container" podUID="5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" containerName="main"
Nov 02 12:33:26.265226 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.265179    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" containerName="main"
Nov 02 12:33:26.265324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.265290    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="5dfe9fdf-8c30-4c80-b3e1-76bf844ae212" containerName="main"
Nov 02 12:33:26.268627 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.268608    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:33:26.323591 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.323542    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-257e6d-serving-cert\""
Nov 02 12:33:26.359673 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.359648    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/107ae880-4744-43a3-a3cb-b90e46faadb6-maas-api-tls\") pod \"maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg\" (UID: \"107ae880-4744-43a3-a3cb-b90e46faadb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:33:26.359825 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.359728    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/107ae880-4744-43a3-a3cb-b90e46faadb6-openshift-service-ca\") pod \"maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg\" (UID: \"107ae880-4744-43a3-a3cb-b90e46faadb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:33:26.359825 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.359822    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/107ae880-4744-43a3-a3cb-b90e46faadb6-metrics-certs\") pod \"maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg\" (UID: \"107ae880-4744-43a3-a3cb-b90e46faadb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:33:26.359957 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.359949    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szvtx\" (UniqueName: \"kubernetes.io/projected/107ae880-4744-43a3-a3cb-b90e46faadb6-kube-api-access-szvtx\") pod \"maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg\" (UID: \"107ae880-4744-43a3-a3cb-b90e46faadb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:33:26.375870 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.375849    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"]
Nov 02 12:33:26.461538 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.461510    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/107ae880-4744-43a3-a3cb-b90e46faadb6-metrics-certs\") pod \"maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg\" (UID: \"107ae880-4744-43a3-a3cb-b90e46faadb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:33:26.461689 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.461597    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szvtx\" (UniqueName: \"kubernetes.io/projected/107ae880-4744-43a3-a3cb-b90e46faadb6-kube-api-access-szvtx\") pod \"maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg\" (UID: \"107ae880-4744-43a3-a3cb-b90e46faadb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:33:26.461689 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.461627    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/107ae880-4744-43a3-a3cb-b90e46faadb6-maas-api-tls\") pod \"maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg\" (UID: \"107ae880-4744-43a3-a3cb-b90e46faadb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:33:26.461689 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.461649    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/107ae880-4744-43a3-a3cb-b90e46faadb6-openshift-service-ca\") pod \"maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg\" (UID: \"107ae880-4744-43a3-a3cb-b90e46faadb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:33:26.461858 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:33:26.461781    4942 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-serving-cert: secret "maas-api-e2e-api-b-257e6d-serving-cert" not found
Nov 02 12:33:26.461858 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:33:26.461855    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/107ae880-4744-43a3-a3cb-b90e46faadb6-maas-api-tls podName:107ae880-4744-43a3-a3cb-b90e46faadb6 nodeName:}" failed. No retries permitted until 2026-11-02 12:33:26.961834126 +0000 UTC m=+2233.533183876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/107ae880-4744-43a3-a3cb-b90e46faadb6-maas-api-tls") pod "maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg" (UID: "107ae880-4744-43a3-a3cb-b90e46faadb6") : secret "maas-api-e2e-api-b-257e6d-serving-cert" not found
Nov 02 12:33:26.462266 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.462247    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/107ae880-4744-43a3-a3cb-b90e46faadb6-openshift-service-ca\") pod \"maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg\" (UID: \"107ae880-4744-43a3-a3cb-b90e46faadb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:33:26.463847 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.463816    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/107ae880-4744-43a3-a3cb-b90e46faadb6-metrics-certs\") pod \"maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg\" (UID: \"107ae880-4744-43a3-a3cb-b90e46faadb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:33:26.527040 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.526953    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szvtx\" (UniqueName: \"kubernetes.io/projected/107ae880-4744-43a3-a3cb-b90e46faadb6-kube-api-access-szvtx\") pod \"maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg\" (UID: \"107ae880-4744-43a3-a3cb-b90e46faadb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:33:26.967660 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.967628    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/107ae880-4744-43a3-a3cb-b90e46faadb6-maas-api-tls\") pod \"maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg\" (UID: \"107ae880-4744-43a3-a3cb-b90e46faadb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:33:26.969972 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:26.969950    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/107ae880-4744-43a3-a3cb-b90e46faadb6-maas-api-tls\") pod \"maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg\" (UID: \"107ae880-4744-43a3-a3cb-b90e46faadb6\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:33:27.182514 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:27.182481    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:33:27.560587 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:27.560407    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"]
Nov 02 12:33:27.563292 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:33:27.563264    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod107ae880_4744_43a3_a3cb_b90e46faadb6.slice/crio-c405575835fbee5c337c1aee9ee7695eaeba003839d6afb73aad2623a8806b97 WatchSource:0}: Error finding container c405575835fbee5c337c1aee9ee7695eaeba003839d6afb73aad2623a8806b97: Status 404 returned error can't find the container with id c405575835fbee5c337c1aee9ee7695eaeba003839d6afb73aad2623a8806b97
Nov 02 12:33:28.232080 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:28.232039    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg" event={"ID":"107ae880-4744-43a3-a3cb-b90e46faadb6","Type":"ContainerStarted","Data":"c405575835fbee5c337c1aee9ee7695eaeba003839d6afb73aad2623a8806b97"}
Nov 02 12:33:29.240933 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:29.240897    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg" event={"ID":"107ae880-4744-43a3-a3cb-b90e46faadb6","Type":"ContainerStarted","Data":"aa94dd7d323dc07e2c92f0158c488d388a8d4cc8e271d204196b9853d3dfcba6"}
Nov 02 12:33:29.241457 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:29.240987    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:33:29.360985 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:29.360929    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg" podStartSLOduration=2.410548951 podStartE2EDuration="3.36091344s" totalImagesPullingTime="950.364489ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:33:29.35921314 +0000 UTC m=+2235.930562897" watchObservedRunningTime="2026-11-02 12:33:29.36091344 +0000 UTC m=+2235.932263197"
Nov 02 12:33:30.982323 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:33:30.982267    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:33:35.253260 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:35.253222    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:33:36.779859 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:36.779820    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"]
Nov 02 12:33:36.784767 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:36.784742    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:36.905924 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:36.905886    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-credential-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:36.906113 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:36.906011    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-workload-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:36.906113 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:36.906041    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-envoy\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:36.906113 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:36.906097    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-podinfo\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:36.906237 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:36.906148    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvvnh\" (UniqueName: \"kubernetes.io/projected/f22de0a6-ec9a-411e-b30f-5e45f20c394b-kube-api-access-mvvnh\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:36.906237 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:36.906202    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istiod-ca-cert\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:36.906385 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:36.906240    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-token\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:36.906385 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:36.906266    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-data\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:36.906385 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:36.906320    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-workload-certs\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:36.913173 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:36.913138    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"]
Nov 02 12:33:37.006952 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.006893    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-workload-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.006952 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.006954    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-envoy\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.007227 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.007006    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-podinfo\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.007227 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.007050    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvvnh\" (UniqueName: \"kubernetes.io/projected/f22de0a6-ec9a-411e-b30f-5e45f20c394b-kube-api-access-mvvnh\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.007227 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.007099    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istiod-ca-cert\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.007227 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.007138    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-token\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.007227 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.007163    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-data\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.007227 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.007206    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-workload-certs\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.007550 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.007255    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-credential-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.007979 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.007943    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-workload-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.008375 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.008352    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-credential-socket\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.008602 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.008520    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-data\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.008936 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.008886    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-workload-certs\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.009205 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.009184    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istiod-ca-cert\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.011555 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.011529    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-podinfo\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.011793 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.011769    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-envoy\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.094365 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.094181    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-token\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.096868 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.096819    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvvnh\" (UniqueName: \"kubernetes.io/projected/f22de0a6-ec9a-411e-b30f-5e45f20c394b-kube-api-access-mvvnh\") pod \"e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") " pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.098829 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.098649    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:37.593356 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.592207    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"]
Nov 02 12:33:37.598029 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:33:37.597998    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf22de0a6_ec9a_411e_b30f_5e45f20c394b.slice/crio-ba784ce11b78fea76e9aaa5c05829a0d6178aaf5c57ab1dbc90a61df66a342b6 WatchSource:0}: Error finding container ba784ce11b78fea76e9aaa5c05829a0d6178aaf5c57ab1dbc90a61df66a342b6: Status 404 returned error can't find the container with id ba784ce11b78fea76e9aaa5c05829a0d6178aaf5c57ab1dbc90a61df66a342b6
Nov 02 12:33:37.600691 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.600651    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:33:37.600810 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.600738    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:33:37.600941 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:37.600921    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:33:38.312659 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:38.311115    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"]
Nov 02 12:33:38.312659 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:38.311389    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m" podUID="a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" containerName="istio-proxy" containerID="cri-o://89308f1d9bd71c48fec93559fbc5d56c2c499645572ab713b8d1837cebb02f36" gracePeriod=30
Nov 02 12:33:38.324194 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:38.324159    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c" event={"ID":"f22de0a6-ec9a-411e-b30f-5e45f20c394b","Type":"ContainerStarted","Data":"a125ae4eb29f08675c1654081f178a3717209a8407065b06440f22d171d2217e"}
Nov 02 12:33:38.324353 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:38.324224    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c" event={"ID":"f22de0a6-ec9a-411e-b30f-5e45f20c394b","Type":"ContainerStarted","Data":"ba784ce11b78fea76e9aaa5c05829a0d6178aaf5c57ab1dbc90a61df66a342b6"}
Nov 02 12:33:38.584376 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:38.583555    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c" podStartSLOduration=2.583535001 podStartE2EDuration="2.583535001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:33:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:33:38.577064613 +0000 UTC m=+2245.148414384" watchObservedRunningTime="2026-11-02 12:33:38.583535001 +0000 UTC m=+2245.154884760"
Nov 02 12:33:39.113320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:39.113007    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:39.115626 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:39.115595    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Nov 02 12:33:39.115846 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:39.115816    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c" podUID="f22de0a6-ec9a-411e-b30f-5e45f20c394b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Nov 02 12:33:40.108325 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:40.104407    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Nov 02 12:33:40.108325 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:40.104474    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c" podUID="f22de0a6-ec9a-411e-b30f-5e45f20c394b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Nov 02 12:33:41.100914 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:41.100876    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Nov 02 12:33:41.101198 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:41.101170    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c" podUID="f22de0a6-ec9a-411e-b30f-5e45f20c394b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Nov 02 12:33:42.102322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:42.099678    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Nov 02 12:33:42.102322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:42.099745    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c" podUID="f22de0a6-ec9a-411e-b30f-5e45f20c394b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Nov 02 12:33:43.099271 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.099231    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Nov 02 12:33:43.099483 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.099322    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c" podUID="f22de0a6-ec9a-411e-b30f-5e45f20c394b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Nov 02 12:33:43.648272 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.648243    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:33:43.746407 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.746323    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-credential-socket\") pod \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") "
Nov 02 12:33:43.746565 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.746452    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-kube-api-access-p7dkq\" (UniqueName: \"kubernetes.io/projected/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-kube-api-access-p7dkq\") pod \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") "
Nov 02 12:33:43.746565 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.746521    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-workload-socket\") pod \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") "
Nov 02 12:33:43.746565 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.746533    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-credential-socket" pod "a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" (UID: "a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:33:43.746565 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.746559    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istiod-ca-cert\") pod \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") "
Nov 02 12:33:43.746776 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.746604    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-data\") pod \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") "
Nov 02 12:33:43.746776 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.746659    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-workload-certs\") pod \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") "
Nov 02 12:33:43.746776 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.746704    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-envoy\") pod \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") "
Nov 02 12:33:43.746776 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.746726    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-workload-socket" pod "a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" (UID: "a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:33:43.746776 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.746738    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-podinfo\") pod \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") "
Nov 02 12:33:43.746776 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.746769    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-token\" (UniqueName: \"kubernetes.io/projected/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-token\") pod \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\" (UID: \"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57\") "
Nov 02 12:33:43.747079 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.746919    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-data" pod "a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" (UID: "a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:33:43.747079 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.747014    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istiod-ca-cert" pod "a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" (UID: "a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:33:43.747079 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.747025    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-workload-certs" pod "a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" (UID: "a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:33:43.747311 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.747272    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:33:43.747437 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.747318    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:33:43.747437 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.747335    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:33:43.747437 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.747347    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:33:43.747437 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.747362    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:33:43.749503 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.749478    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-kube-api-access-p7dkq" pod "a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" (UID: "a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57"). InnerVolumeSpecName "kube-api-access-p7dkq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:33:43.749761 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.749736    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-podinfo" pod "a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" (UID: "a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:33:43.749855 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.749839    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-envoy" pod "a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" (UID: "a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:33:43.750543 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.750519    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-token" pod "a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" (UID: "a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:33:43.848907 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.848864    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7dkq\" (UniqueName: \"kubernetes.io/projected/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-kube-api-access-p7dkq\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:33:43.848907 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.848907    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:33:43.849159 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.848924    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:33:43.849159 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:43.848938    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:33:44.100265 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:44.100230    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Nov 02 12:33:44.100437 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:44.100306    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c" podUID="f22de0a6-ec9a-411e-b30f-5e45f20c394b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Nov 02 12:33:44.379592 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:44.379497    4942 generic.go:356] "Generic (PLEG): container finished" podID="a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" containerID="89308f1d9bd71c48fec93559fbc5d56c2c499645572ab713b8d1837cebb02f36" exitCode=0
Nov 02 12:33:44.379592 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:44.379544    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"
Nov 02 12:33:44.379592 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:44.379574    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m" event={"ID":"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57","Type":"ContainerDied","Data":"89308f1d9bd71c48fec93559fbc5d56c2c499645572ab713b8d1837cebb02f36"}
Nov 02 12:33:44.379869 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:44.379629    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m" event={"ID":"a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57","Type":"ContainerDied","Data":"27b61a47b58dd87893600b9496131f5687579330a50c4afd78bedb56eca3c5d6"}
Nov 02 12:33:44.379869 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:44.379690    4942 scope.go:122] "RemoveContainer" containerID="89308f1d9bd71c48fec93559fbc5d56c2c499645572ab713b8d1837cebb02f36"
Nov 02 12:33:44.390947 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:44.390929    4942 scope.go:122] "RemoveContainer" containerID="89308f1d9bd71c48fec93559fbc5d56c2c499645572ab713b8d1837cebb02f36"
Nov 02 12:33:44.391215 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:33:44.391189    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89308f1d9bd71c48fec93559fbc5d56c2c499645572ab713b8d1837cebb02f36\": container with ID starting with 89308f1d9bd71c48fec93559fbc5d56c2c499645572ab713b8d1837cebb02f36 not found: ID does not exist" containerID="89308f1d9bd71c48fec93559fbc5d56c2c499645572ab713b8d1837cebb02f36"
Nov 02 12:33:44.391329 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:44.391225    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89308f1d9bd71c48fec93559fbc5d56c2c499645572ab713b8d1837cebb02f36"} err="failed to get container status \"89308f1d9bd71c48fec93559fbc5d56c2c499645572ab713b8d1837cebb02f36\": rpc error: code = NotFound desc = could not find container \"89308f1d9bd71c48fec93559fbc5d56c2c499645572ab713b8d1837cebb02f36\": container with ID starting with 89308f1d9bd71c48fec93559fbc5d56c2c499645572ab713b8d1837cebb02f36 not found: ID does not exist"
Nov 02 12:33:44.547217 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:44.547147    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"]
Nov 02 12:33:44.550963 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:44.550930    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-9gj2m"]
Nov 02 12:33:45.099861 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:45.099265    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body=
Nov 02 12:33:45.099861 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:45.099360    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c" podUID="f22de0a6-ec9a-411e-b30f-5e45f20c394b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused"
Nov 02 12:33:45.547222 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:45.547142    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" path="/var/lib/kubelet/pods/a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57/volumes"
Nov 02 12:33:46.312125 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:46.308702    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:46.312125 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:46.310064    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:46.318082 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:46.318049    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:33:47.410892 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:47.410857    4942 generic.go:356] "Generic (PLEG): container finished" podID="5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" containerID="b6cb99aa29c097599373493eb0427d013e05c0bc8d1b672387298edb6f716407" exitCode=0
Nov 02 12:33:47.411384 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:47.410934    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4" event={"ID":"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c","Type":"ContainerDied","Data":"b6cb99aa29c097599373493eb0427d013e05c0bc8d1b672387298edb6f716407"}
Nov 02 12:33:47.411480 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:47.411394    4942 scope.go:122] "RemoveContainer" containerID="b6cb99aa29c097599373493eb0427d013e05c0bc8d1b672387298edb6f716407"
Nov 02 12:33:47.412705 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:47.412668    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:33:47.412819 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:47.412761    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:33:47.412892 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:47.412814    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:33:47.977530 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:47.977411    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:33:48.421207 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:48.421166    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4" event={"ID":"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c","Type":"ContainerStarted","Data":"460014b0dc2abb36e6461ea45a2a22bb28be899a2e14f5493ba80328dd869248"}
Nov 02 12:33:48.636920 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:48.636888    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:33:48.637957 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:48.637931    4942 patch_prober.go:29] interesting pod/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Nov 02 12:33:48.638082 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:48.637978    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4" podUID="5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Nov 02 12:33:49.637406 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:49.637367    4942 patch_prober.go:29] interesting pod/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Nov 02 12:33:49.637876 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:49.637430    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4" podUID="5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Nov 02 12:33:50.637322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:50.636665    4942 patch_prober.go:29] interesting pod/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Nov 02 12:33:50.637322 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:50.636729    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4" podUID="5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Nov 02 12:33:51.637306 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:51.637270    4942 patch_prober.go:29] interesting pod/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Nov 02 12:33:51.637487 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:51.637365    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4" podUID="5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Nov 02 12:33:52.637088 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:52.637046    4942 patch_prober.go:29] interesting pod/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused" start-of-body=
Nov 02 12:33:52.637609 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:52.637111    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4" podUID="5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.107:15021/healthz/ready\": dial tcp 10.132.0.107:15021: connect: connection refused"
Nov 02 12:33:53.640663 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:53.640633    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:33:53.641067 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:53.641023    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:33:53.641729 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:53.641707    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:33:59.167865 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:59.167834    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"]
Nov 02 12:33:59.168349 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:59.168034    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c" podUID="f22de0a6-ec9a-411e-b30f-5e45f20c394b" containerName="istio-proxy" containerID="cri-o://a125ae4eb29f08675c1654081f178a3717209a8407065b06440f22d171d2217e" gracePeriod=30
Nov 02 12:33:59.182750 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:59.182724    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"]
Nov 02 12:33:59.182932 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:33:59.182899    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m" podUID="feca8edf-3c6e-4370-8e3c-c68e195c655c" containerName="istio-proxy" containerID="cri-o://9a00fff8534f3c669c450142d6a88b1eae803bb39131ac0bbf5ec126a3a471fb" gracePeriod=30
Nov 02 12:34:00.984752 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:34:00.984710    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:34:01.312635 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:01.312556    4942 patch_prober.go:29] interesting pod/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 12:34:01.312635 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:01.312609    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c" podUID="f22de0a6-ec9a-411e-b30f-5e45f20c394b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 12:34:04.454238 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.454207    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:34:04.487286 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.487257    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:34:04.517108 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517078    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-credential-socket\") pod \"feca8edf-3c6e-4370-8e3c-c68e195c655c\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") "
Nov 02 12:34:04.517247 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517114    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-data\") pod \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") "
Nov 02 12:34:04.517247 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517143    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-credential-socket\") pod \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") "
Nov 02 12:34:04.517247 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517168    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-envoy\") pod \"feca8edf-3c6e-4370-8e3c-c68e195c655c\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") "
Nov 02 12:34:04.517247 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517204    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istiod-ca-cert\") pod \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") "
Nov 02 12:34:04.517518 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517262    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f22de0a6-ec9a-411e-b30f-5e45f20c394b-kube-api-access-mvvnh\" (UniqueName: \"kubernetes.io/projected/f22de0a6-ec9a-411e-b30f-5e45f20c394b-kube-api-access-mvvnh\") pod \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") "
Nov 02 12:34:04.517518 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517329    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-workload-certs\") pod \"feca8edf-3c6e-4370-8e3c-c68e195c655c\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") "
Nov 02 12:34:04.517518 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517357    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-workload-socket\") pod \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") "
Nov 02 12:34:04.517518 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517390    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-data\") pod \"feca8edf-3c6e-4370-8e3c-c68e195c655c\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") "
Nov 02 12:34:04.517518 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517394    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-data" pod "f22de0a6-ec9a-411e-b30f-5e45f20c394b" (UID: "f22de0a6-ec9a-411e-b30f-5e45f20c394b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:34:04.517518 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517431    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-podinfo\") pod \"feca8edf-3c6e-4370-8e3c-c68e195c655c\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") "
Nov 02 12:34:04.517518 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517482    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-envoy\") pod \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") "
Nov 02 12:34:04.517858 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517577    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/feca8edf-3c6e-4370-8e3c-c68e195c655c-kube-api-access-5sn8r\" (UniqueName: \"kubernetes.io/projected/feca8edf-3c6e-4370-8e3c-c68e195c655c-kube-api-access-5sn8r\") pod \"feca8edf-3c6e-4370-8e3c-c68e195c655c\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") "
Nov 02 12:34:04.517858 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517624    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-token\" (UniqueName: \"kubernetes.io/projected/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-token\") pod \"feca8edf-3c6e-4370-8e3c-c68e195c655c\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") "
Nov 02 12:34:04.517858 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517676    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-workload-socket\") pod \"feca8edf-3c6e-4370-8e3c-c68e195c655c\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") "
Nov 02 12:34:04.517858 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517714    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-token\" (UniqueName: \"kubernetes.io/projected/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-token\") pod \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") "
Nov 02 12:34:04.517858 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517750    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-podinfo\") pod \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") "
Nov 02 12:34:04.517858 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517790    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-workload-certs\") pod \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\" (UID: \"f22de0a6-ec9a-411e-b30f-5e45f20c394b\") "
Nov 02 12:34:04.517858 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517826    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/feca8edf-3c6e-4370-8e3c-c68e195c655c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/feca8edf-3c6e-4370-8e3c-c68e195c655c-istiod-ca-cert\") pod \"feca8edf-3c6e-4370-8e3c-c68e195c655c\" (UID: \"feca8edf-3c6e-4370-8e3c-c68e195c655c\") "
Nov 02 12:34:04.518207 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517403    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-credential-socket" pod "feca8edf-3c6e-4370-8e3c-c68e195c655c" (UID: "feca8edf-3c6e-4370-8e3c-c68e195c655c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:34:04.518207 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517453    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-credential-socket" pod "f22de0a6-ec9a-411e-b30f-5e45f20c394b" (UID: "f22de0a6-ec9a-411e-b30f-5e45f20c394b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:34:04.518207 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517916    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-workload-socket" pod "f22de0a6-ec9a-411e-b30f-5e45f20c394b" (UID: "f22de0a6-ec9a-411e-b30f-5e45f20c394b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:34:04.518207 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517979    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-workload-certs" pod "feca8edf-3c6e-4370-8e3c-c68e195c655c" (UID: "feca8edf-3c6e-4370-8e3c-c68e195c655c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:34:04.518207 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.517992    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-data" pod "feca8edf-3c6e-4370-8e3c-c68e195c655c" (UID: "feca8edf-3c6e-4370-8e3c-c68e195c655c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:34:04.518478 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.518265    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-workload-socket" pod "feca8edf-3c6e-4370-8e3c-c68e195c655c" (UID: "feca8edf-3c6e-4370-8e3c-c68e195c655c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:34:04.518478 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.518329    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-workload-certs" pod "f22de0a6-ec9a-411e-b30f-5e45f20c394b" (UID: "f22de0a6-ec9a-411e-b30f-5e45f20c394b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:34:04.518478 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.518376    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istiod-ca-cert" pod "f22de0a6-ec9a-411e-b30f-5e45f20c394b" (UID: "f22de0a6-ec9a-411e-b30f-5e45f20c394b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:34:04.518628 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.518578    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/feca8edf-3c6e-4370-8e3c-c68e195c655c-istiod-ca-cert" pod "feca8edf-3c6e-4370-8e3c-c68e195c655c" (UID: "feca8edf-3c6e-4370-8e3c-c68e195c655c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:34:04.518949 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.518928    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:04.519037 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.518955    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:04.519037 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.518970    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:04.519037 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.518989    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:04.519037 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.519003    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:04.519037 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.519017    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:04.519037 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.519031    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:04.519402 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.519045    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:04.519402 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.519059    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:04.519402 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.519074    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/feca8edf-3c6e-4370-8e3c-c68e195c655c-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:04.519742 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.519714    4942 generic.go:356] "Generic (PLEG): container finished" podID="feca8edf-3c6e-4370-8e3c-c68e195c655c" containerID="9a00fff8534f3c669c450142d6a88b1eae803bb39131ac0bbf5ec126a3a471fb" exitCode=0
Nov 02 12:34:04.519851 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.519832    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m" event={"ID":"feca8edf-3c6e-4370-8e3c-c68e195c655c","Type":"ContainerDied","Data":"9a00fff8534f3c669c450142d6a88b1eae803bb39131ac0bbf5ec126a3a471fb"}
Nov 02 12:34:04.519932 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.519882    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m" event={"ID":"feca8edf-3c6e-4370-8e3c-c68e195c655c","Type":"ContainerDied","Data":"d93a4f7c04741b69883e3740ead6dbcfe8666ea4040bda090b46c1818fdbaf2a"}
Nov 02 12:34:04.519932 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.519915    4942 scope.go:122] "RemoveContainer" containerID="9a00fff8534f3c669c450142d6a88b1eae803bb39131ac0bbf5ec126a3a471fb"
Nov 02 12:34:04.520096 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.520081    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"
Nov 02 12:34:04.520292 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.520268    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-podinfo" pod "feca8edf-3c6e-4370-8e3c-c68e195c655c" (UID: "feca8edf-3c6e-4370-8e3c-c68e195c655c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:34:04.520744 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.520712    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-token" pod "feca8edf-3c6e-4370-8e3c-c68e195c655c" (UID: "feca8edf-3c6e-4370-8e3c-c68e195c655c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:34:04.520835 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.520748    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-envoy" pod "feca8edf-3c6e-4370-8e3c-c68e195c655c" (UID: "feca8edf-3c6e-4370-8e3c-c68e195c655c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:34:04.520835 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.520752    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-token" pod "f22de0a6-ec9a-411e-b30f-5e45f20c394b" (UID: "f22de0a6-ec9a-411e-b30f-5e45f20c394b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:34:04.520835 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.520766    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feca8edf-3c6e-4370-8e3c-c68e195c655c-kube-api-access-5sn8r" pod "feca8edf-3c6e-4370-8e3c-c68e195c655c" (UID: "feca8edf-3c6e-4370-8e3c-c68e195c655c"). InnerVolumeSpecName "kube-api-access-5sn8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:34:04.522409 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.521892    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f22de0a6-ec9a-411e-b30f-5e45f20c394b-kube-api-access-mvvnh" pod "f22de0a6-ec9a-411e-b30f-5e45f20c394b" (UID: "f22de0a6-ec9a-411e-b30f-5e45f20c394b"). InnerVolumeSpecName "kube-api-access-mvvnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:34:04.522945 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.522919    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-envoy" pod "f22de0a6-ec9a-411e-b30f-5e45f20c394b" (UID: "f22de0a6-ec9a-411e-b30f-5e45f20c394b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:34:04.524454 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.524434    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"
Nov 02 12:34:04.524874 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.524672    4942 generic.go:356] "Generic (PLEG): container finished" podID="f22de0a6-ec9a-411e-b30f-5e45f20c394b" containerID="a125ae4eb29f08675c1654081f178a3717209a8407065b06440f22d171d2217e" exitCode=0
Nov 02 12:34:04.524874 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.524792    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c" event={"ID":"f22de0a6-ec9a-411e-b30f-5e45f20c394b","Type":"ContainerDied","Data":"a125ae4eb29f08675c1654081f178a3717209a8407065b06440f22d171d2217e"}
Nov 02 12:34:04.524874 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.524829    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c" event={"ID":"f22de0a6-ec9a-411e-b30f-5e45f20c394b","Type":"ContainerDied","Data":"ba784ce11b78fea76e9aaa5c05829a0d6178aaf5c57ab1dbc90a61df66a342b6"}
Nov 02 12:34:04.526226 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.526207    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-podinfo" pod "f22de0a6-ec9a-411e-b30f-5e45f20c394b" (UID: "f22de0a6-ec9a-411e-b30f-5e45f20c394b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:34:04.532938 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.532918    4942 scope.go:122] "RemoveContainer" containerID="9a00fff8534f3c669c450142d6a88b1eae803bb39131ac0bbf5ec126a3a471fb"
Nov 02 12:34:04.533205 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:34:04.533186    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a00fff8534f3c669c450142d6a88b1eae803bb39131ac0bbf5ec126a3a471fb\": container with ID starting with 9a00fff8534f3c669c450142d6a88b1eae803bb39131ac0bbf5ec126a3a471fb not found: ID does not exist" containerID="9a00fff8534f3c669c450142d6a88b1eae803bb39131ac0bbf5ec126a3a471fb"
Nov 02 12:34:04.533255 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.533215    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a00fff8534f3c669c450142d6a88b1eae803bb39131ac0bbf5ec126a3a471fb"} err="failed to get container status \"9a00fff8534f3c669c450142d6a88b1eae803bb39131ac0bbf5ec126a3a471fb\": rpc error: code = NotFound desc = could not find container \"9a00fff8534f3c669c450142d6a88b1eae803bb39131ac0bbf5ec126a3a471fb\": container with ID starting with 9a00fff8534f3c669c450142d6a88b1eae803bb39131ac0bbf5ec126a3a471fb not found: ID does not exist"
Nov 02 12:34:04.533255 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.533235    4942 scope.go:122] "RemoveContainer" containerID="a125ae4eb29f08675c1654081f178a3717209a8407065b06440f22d171d2217e"
Nov 02 12:34:04.541546 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.541526    4942 scope.go:122] "RemoveContainer" containerID="a125ae4eb29f08675c1654081f178a3717209a8407065b06440f22d171d2217e"
Nov 02 12:34:04.541789 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:34:04.541772    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a125ae4eb29f08675c1654081f178a3717209a8407065b06440f22d171d2217e\": container with ID starting with a125ae4eb29f08675c1654081f178a3717209a8407065b06440f22d171d2217e not found: ID does not exist" containerID="a125ae4eb29f08675c1654081f178a3717209a8407065b06440f22d171d2217e"
Nov 02 12:34:04.541837 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.541795    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a125ae4eb29f08675c1654081f178a3717209a8407065b06440f22d171d2217e"} err="failed to get container status \"a125ae4eb29f08675c1654081f178a3717209a8407065b06440f22d171d2217e\": rpc error: code = NotFound desc = could not find container \"a125ae4eb29f08675c1654081f178a3717209a8407065b06440f22d171d2217e\": container with ID starting with a125ae4eb29f08675c1654081f178a3717209a8407065b06440f22d171d2217e not found: ID does not exist"
Nov 02 12:34:04.619973 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.619936    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:04.619973 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.619973    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:04.620181 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.619989    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:04.620181 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.620004    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvvnh\" (UniqueName: \"kubernetes.io/projected/f22de0a6-ec9a-411e-b30f-5e45f20c394b-kube-api-access-mvvnh\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:04.620181 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.620019    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:04.620181 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.620033    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f22de0a6-ec9a-411e-b30f-5e45f20c394b-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:04.620181 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.620048    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sn8r\" (UniqueName: \"kubernetes.io/projected/feca8edf-3c6e-4370-8e3c-c68e195c655c-kube-api-access-5sn8r\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:04.620181 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.620061    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/feca8edf-3c6e-4370-8e3c-c68e195c655c-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:04.941859 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.941811    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"]
Nov 02 12:34:04.945553 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:04.945527    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-g469m"]
Nov 02 12:34:05.054377 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:05.054336    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"]
Nov 02 12:34:05.060681 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:05.060648    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-4b9210-openshift-default-dcf4c64c-mwb4c"]
Nov 02 12:34:05.543386 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:05.543354    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f22de0a6-ec9a-411e-b30f-5e45f20c394b" path="/var/lib/kubelet/pods/f22de0a6-ec9a-411e-b30f-5e45f20c394b/volumes"
Nov 02 12:34:05.543998 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:05.543977    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="feca8edf-3c6e-4370-8e3c-c68e195c655c" path="/var/lib/kubelet/pods/feca8edf-3c6e-4370-8e3c-c68e195c655c/volumes"
Nov 02 12:34:09.046034 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.045995    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"]
Nov 02 12:34:09.046789 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.046765    4942 cpu_manager.go:414] "containerMap: removing container" podUID="a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" containerName="istio-proxy"
Nov 02 12:34:09.046843 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.046792    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" containerName="istio-proxy"
Nov 02 12:34:09.046843 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.046815    4942 cpu_manager.go:414] "containerMap: removing container" podUID="f22de0a6-ec9a-411e-b30f-5e45f20c394b" containerName="istio-proxy"
Nov 02 12:34:09.046843 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.046823    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f22de0a6-ec9a-411e-b30f-5e45f20c394b" containerName="istio-proxy"
Nov 02 12:34:09.046954 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.046856    4942 cpu_manager.go:414] "containerMap: removing container" podUID="feca8edf-3c6e-4370-8e3c-c68e195c655c" containerName="istio-proxy"
Nov 02 12:34:09.046954 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.046865    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="feca8edf-3c6e-4370-8e3c-c68e195c655c" containerName="istio-proxy"
Nov 02 12:34:09.047029 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.046995    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="feca8edf-3c6e-4370-8e3c-c68e195c655c" containerName="istio-proxy"
Nov 02 12:34:09.047029 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.047013    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4cfa7a7-872f-45d8-aacd-e7fca4cc2e57" containerName="istio-proxy"
Nov 02 12:34:09.047029 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.047019    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="f22de0a6-ec9a-411e-b30f-5e45f20c394b" containerName="istio-proxy"
Nov 02 12:34:09.050421 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.050400    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.107334 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.102670    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-6373b9-openshift-default-dockercfg-jds74\""
Nov 02 12:34:09.155644 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.155615    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"]
Nov 02 12:34:09.174972 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.174941    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-envoy\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.175107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.174975    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-data\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.175107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.174994    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-podinfo\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.175107 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.175090    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-workload-socket\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.175238 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.175141    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx8vt\" (UniqueName: \"kubernetes.io/projected/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-kube-api-access-jx8vt\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.175238 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.175160    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-token\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.175365 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.175318    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-credential-socket\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.175421 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.175395    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-workload-certs\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.175475 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.175425    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istiod-ca-cert\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.276560 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.276522    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-envoy\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.276742 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.276573    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-data\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.276742 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.276605    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-podinfo\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.276742 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.276648    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-workload-socket\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.276742 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.276691    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx8vt\" (UniqueName: \"kubernetes.io/projected/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-kube-api-access-jx8vt\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.276742 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.276716    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-token\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.277051 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.276797    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-credential-socket\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.277051 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.276864    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-workload-certs\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.277051 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.276887    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istiod-ca-cert\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.277051 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.277036    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-data\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.277272 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.277090    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-workload-socket\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.277355 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.277324    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-credential-socket\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.277619 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.277591    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-workload-certs\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.277693 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.277677    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istiod-ca-cert\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.279408 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.279378    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-envoy\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.279536 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.279381    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-podinfo\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.337308 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.337238    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-token\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.337565 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.337543    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx8vt\" (UniqueName: \"kubernetes.io/projected/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-kube-api-access-jx8vt\") pod \"e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") " pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.366391 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.366355    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:09.798205 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.798172    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"]
Nov 02 12:34:09.799279 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:34:09.799254    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29c6e230_f0a0_4d1f_bb69_db84edee3f9e.slice/crio-93b1910ae2fca0be70b10fe09e1109a4725618759a151e53fa2026878fab0a29 WatchSource:0}: Error finding container 93b1910ae2fca0be70b10fe09e1109a4725618759a151e53fa2026878fab0a29: Status 404 returned error can't find the container with id 93b1910ae2fca0be70b10fe09e1109a4725618759a151e53fa2026878fab0a29
Nov 02 12:34:09.801632 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.801596    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:34:09.801712 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.801664    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:34:09.801712 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:09.801695    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:34:10.566395 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:10.566353    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk" event={"ID":"29c6e230-f0a0-4d1f-bb69-db84edee3f9e","Type":"ContainerStarted","Data":"e1cff1c42c764ac4c9d4afa98fc6f5ac1cd0e51e6dad64e14a2d2a2ae5b10080"}
Nov 02 12:34:10.566818 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:10.566416    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk" event={"ID":"29c6e230-f0a0-4d1f-bb69-db84edee3f9e","Type":"ContainerStarted","Data":"93b1910ae2fca0be70b10fe09e1109a4725618759a151e53fa2026878fab0a29"}
Nov 02 12:34:10.687781 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:10.687724    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk" podStartSLOduration=1.6877112269999999 podStartE2EDuration="1.687711227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:34:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:34:10.685409636 +0000 UTC m=+2277.256759396" watchObservedRunningTime="2026-11-02 12:34:10.687711227 +0000 UTC m=+2277.259060983"
Nov 02 12:34:11.366875 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:11.366834    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:11.372314 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:11.372273    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:11.573311 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:11.573274    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:11.574132 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:11.574116    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:34:30.984677 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:34:30.984591    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:34:46.667341 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:46.665038    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"]
Nov 02 12:34:46.674470 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:46.674442    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:34:46.727182 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:46.727143    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"]
Nov 02 12:34:46.772768 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:46.772735    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-6373b9-serving-cert\""
Nov 02 12:34:46.802792 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:46.802762    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krzb9\" (UniqueName: \"kubernetes.io/projected/6144b552-eded-47c0-97a8-800af7f06e67-kube-api-access-krzb9\") pod \"maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh\" (UID: \"6144b552-eded-47c0-97a8-800af7f06e67\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:34:46.802967 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:46.802817    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6144b552-eded-47c0-97a8-800af7f06e67-metrics-certs\") pod \"maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh\" (UID: \"6144b552-eded-47c0-97a8-800af7f06e67\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:34:46.802967 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:46.802884    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6144b552-eded-47c0-97a8-800af7f06e67-maas-api-tls\") pod \"maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh\" (UID: \"6144b552-eded-47c0-97a8-800af7f06e67\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:34:46.802967 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:46.802957    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6144b552-eded-47c0-97a8-800af7f06e67-openshift-service-ca\") pod \"maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh\" (UID: \"6144b552-eded-47c0-97a8-800af7f06e67\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:34:46.903841 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:46.903805    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krzb9\" (UniqueName: \"kubernetes.io/projected/6144b552-eded-47c0-97a8-800af7f06e67-kube-api-access-krzb9\") pod \"maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh\" (UID: \"6144b552-eded-47c0-97a8-800af7f06e67\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:34:46.904007 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:46.903848    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6144b552-eded-47c0-97a8-800af7f06e67-metrics-certs\") pod \"maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh\" (UID: \"6144b552-eded-47c0-97a8-800af7f06e67\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:34:46.904055 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:46.903995    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6144b552-eded-47c0-97a8-800af7f06e67-maas-api-tls\") pod \"maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh\" (UID: \"6144b552-eded-47c0-97a8-800af7f06e67\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:34:46.904108 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:46.904090    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6144b552-eded-47c0-97a8-800af7f06e67-openshift-service-ca\") pod \"maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh\" (UID: \"6144b552-eded-47c0-97a8-800af7f06e67\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:34:46.904197 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:34:46.904173    4942 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-serving-cert: secret "maas-api-e2e-explicit-6373b9-serving-cert" not found
Nov 02 12:34:46.904375 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:34:46.904359    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6144b552-eded-47c0-97a8-800af7f06e67-maas-api-tls podName:6144b552-eded-47c0-97a8-800af7f06e67 nodeName:}" failed. No retries permitted until 2026-11-02 12:34:47.40433714 +0000 UTC m=+2313.975686885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6144b552-eded-47c0-97a8-800af7f06e67-maas-api-tls") pod "maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh" (UID: "6144b552-eded-47c0-97a8-800af7f06e67") : secret "maas-api-e2e-explicit-6373b9-serving-cert" not found
Nov 02 12:34:46.904798 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:46.904778    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6144b552-eded-47c0-97a8-800af7f06e67-openshift-service-ca\") pod \"maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh\" (UID: \"6144b552-eded-47c0-97a8-800af7f06e67\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:34:46.906539 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:46.906519    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6144b552-eded-47c0-97a8-800af7f06e67-metrics-certs\") pod \"maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh\" (UID: \"6144b552-eded-47c0-97a8-800af7f06e67\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:34:46.967255 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:46.967181    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krzb9\" (UniqueName: \"kubernetes.io/projected/6144b552-eded-47c0-97a8-800af7f06e67-kube-api-access-krzb9\") pod \"maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh\" (UID: \"6144b552-eded-47c0-97a8-800af7f06e67\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:34:47.411118 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:47.411086    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6144b552-eded-47c0-97a8-800af7f06e67-maas-api-tls\") pod \"maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh\" (UID: \"6144b552-eded-47c0-97a8-800af7f06e67\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:34:47.413754 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:47.413724    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6144b552-eded-47c0-97a8-800af7f06e67-maas-api-tls\") pod \"maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh\" (UID: \"6144b552-eded-47c0-97a8-800af7f06e67\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:34:47.588055 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:47.588023    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:34:47.966721 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:47.966694    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"]
Nov 02 12:34:47.967659 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:34:47.967634    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6144b552_eded_47c0_97a8_800af7f06e67.slice/crio-4b61db88eab28987578f7c79aa0d31b83c6730205caf92c3c001e5a320428256 WatchSource:0}: Error finding container 4b61db88eab28987578f7c79aa0d31b83c6730205caf92c3c001e5a320428256: Status 404 returned error can't find the container with id 4b61db88eab28987578f7c79aa0d31b83c6730205caf92c3c001e5a320428256
Nov 02 12:34:48.834316 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:48.834273    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh" event={"ID":"6144b552-eded-47c0-97a8-800af7f06e67","Type":"ContainerStarted","Data":"4b61db88eab28987578f7c79aa0d31b83c6730205caf92c3c001e5a320428256"}
Nov 02 12:34:49.262032 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:49.261988    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4"]
Nov 02 12:34:49.860456 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:49.860414    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh" event={"ID":"6144b552-eded-47c0-97a8-800af7f06e67","Type":"ContainerStarted","Data":"5d30562d26c39a56637fe86f1e52a2b5bb4f4fe8bed9dca00e27e0295716b351"}
Nov 02 12:34:49.860669 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:49.860506    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:34:49.980758 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:49.980697    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh" podStartSLOduration=3.033856447 podStartE2EDuration="3.980678915s" totalImagesPullingTime="946.822468ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:34:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:34:49.977189559 +0000 UTC m=+2316.548539317" watchObservedRunningTime="2026-11-02 12:34:49.980678915 +0000 UTC m=+2316.552028674"
Nov 02 12:34:54.262366 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:54.262324    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4" podUID="d61ef5ad-d084-49a0-8241-eb808e8df109" containerName="payload-pre-processing" containerID="cri-o://a7d15820d1ec158eeef7b1c81d8980228f4f320777e316b21f131fd122efbcbd" gracePeriod=25
Nov 02 12:34:54.534105 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:54.534077    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4"
Nov 02 12:34:54.688875 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:54.688837    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d61ef5ad-d084-49a0-8241-eb808e8df109-kube-api-access-bwbfl\" (UniqueName: \"kubernetes.io/projected/d61ef5ad-d084-49a0-8241-eb808e8df109-kube-api-access-bwbfl\") pod \"d61ef5ad-d084-49a0-8241-eb808e8df109\" (UID: \"d61ef5ad-d084-49a0-8241-eb808e8df109\") "
Nov 02 12:34:54.689061 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:54.688977    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d61ef5ad-d084-49a0-8241-eb808e8df109-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d61ef5ad-d084-49a0-8241-eb808e8df109-plugins-config-volume\") pod \"d61ef5ad-d084-49a0-8241-eb808e8df109\" (UID: \"d61ef5ad-d084-49a0-8241-eb808e8df109\") "
Nov 02 12:34:54.689498 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:54.689424    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d61ef5ad-d084-49a0-8241-eb808e8df109-plugins-config-volume" pod "d61ef5ad-d084-49a0-8241-eb808e8df109" (UID: "d61ef5ad-d084-49a0-8241-eb808e8df109"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:34:54.691025 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:54.690998    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d61ef5ad-d084-49a0-8241-eb808e8df109-kube-api-access-bwbfl" pod "d61ef5ad-d084-49a0-8241-eb808e8df109" (UID: "d61ef5ad-d084-49a0-8241-eb808e8df109"). InnerVolumeSpecName "kube-api-access-bwbfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:34:54.790919 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:54.790857    4942 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d61ef5ad-d084-49a0-8241-eb808e8df109-plugins-config-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:54.790919 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:54.790884    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwbfl\" (UniqueName: \"kubernetes.io/projected/d61ef5ad-d084-49a0-8241-eb808e8df109-kube-api-access-bwbfl\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:34:54.895156 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:54.895125    4942 generic.go:356] "Generic (PLEG): container finished" podID="d61ef5ad-d084-49a0-8241-eb808e8df109" containerID="a7d15820d1ec158eeef7b1c81d8980228f4f320777e316b21f131fd122efbcbd" exitCode=0
Nov 02 12:34:54.895314 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:54.895169    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4"
Nov 02 12:34:54.895314 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:54.895209    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4" event={"ID":"d61ef5ad-d084-49a0-8241-eb808e8df109","Type":"ContainerDied","Data":"a7d15820d1ec158eeef7b1c81d8980228f4f320777e316b21f131fd122efbcbd"}
Nov 02 12:34:54.895314 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:54.895260    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4" event={"ID":"d61ef5ad-d084-49a0-8241-eb808e8df109","Type":"ContainerDied","Data":"d7d899814dd683c2910a9abea5d9f615cf21b76b41f2cd415c8c935c86417565"}
Nov 02 12:34:54.895314 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:54.895289    4942 scope.go:122] "RemoveContainer" containerID="a7d15820d1ec158eeef7b1c81d8980228f4f320777e316b21f131fd122efbcbd"
Nov 02 12:34:54.906111 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:54.906093    4942 scope.go:122] "RemoveContainer" containerID="a7d15820d1ec158eeef7b1c81d8980228f4f320777e316b21f131fd122efbcbd"
Nov 02 12:34:54.906403 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:34:54.906385    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7d15820d1ec158eeef7b1c81d8980228f4f320777e316b21f131fd122efbcbd\": container with ID starting with a7d15820d1ec158eeef7b1c81d8980228f4f320777e316b21f131fd122efbcbd not found: ID does not exist" containerID="a7d15820d1ec158eeef7b1c81d8980228f4f320777e316b21f131fd122efbcbd"
Nov 02 12:34:54.906482 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:54.906410    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7d15820d1ec158eeef7b1c81d8980228f4f320777e316b21f131fd122efbcbd"} err="failed to get container status \"a7d15820d1ec158eeef7b1c81d8980228f4f320777e316b21f131fd122efbcbd\": rpc error: code = NotFound desc = could not find container \"a7d15820d1ec158eeef7b1c81d8980228f4f320777e316b21f131fd122efbcbd\": container with ID starting with a7d15820d1ec158eeef7b1c81d8980228f4f320777e316b21f131fd122efbcbd not found: ID does not exist"
Nov 02 12:34:55.059575 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:55.059486    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4"]
Nov 02 12:34:55.063227 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:55.063199    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-257e6d-5c49bdd555-jgqm4"]
Nov 02 12:34:55.542889 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:55.542847    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d61ef5ad-d084-49a0-8241-eb808e8df109" path="/var/lib/kubelet/pods/d61ef5ad-d084-49a0-8241-eb808e8df109/volumes"
Nov 02 12:34:55.871891 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:34:55.871862    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:35:01.006188 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:35:01.006148    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:35:05.965924 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:05.965893    4942 generic.go:356] "Generic (PLEG): container finished" podID="29c6e230-f0a0-4d1f-bb69-db84edee3f9e" containerID="e1cff1c42c764ac4c9d4afa98fc6f5ac1cd0e51e6dad64e14a2d2a2ae5b10080" exitCode=0
Nov 02 12:35:05.966291 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:05.965961    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk" event={"ID":"29c6e230-f0a0-4d1f-bb69-db84edee3f9e","Type":"ContainerDied","Data":"e1cff1c42c764ac4c9d4afa98fc6f5ac1cd0e51e6dad64e14a2d2a2ae5b10080"}
Nov 02 12:35:05.966361 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:05.966336    4942 scope.go:122] "RemoveContainer" containerID="e1cff1c42c764ac4c9d4afa98fc6f5ac1cd0e51e6dad64e14a2d2a2ae5b10080"
Nov 02 12:35:05.967650 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:05.967620    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:35:05.967721 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:05.967700    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:35:05.967766 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:05.967750    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 12:35:06.984447 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:06.984411    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk" event={"ID":"29c6e230-f0a0-4d1f-bb69-db84edee3f9e","Type":"ContainerStarted","Data":"d98c80a01e7049ba42c410fecb46f3ad6ba15cc7510db119e2581fff1176f93c"}
Nov 02 12:35:07.371324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:07.370537    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:35:07.371324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:07.370680    4942 patch_prober.go:29] interesting pod/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 02 12:35:07.371324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:07.370726    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk" podUID="29c6e230-f0a0-4d1f-bb69-db84edee3f9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 02 12:35:08.371709 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:08.371664    4942 patch_prober.go:29] interesting pod/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 02 12:35:08.373357 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:08.373325    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk" podUID="29c6e230-f0a0-4d1f-bb69-db84edee3f9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 02 12:35:09.368321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:09.367701    4942 patch_prober.go:29] interesting pod/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 02 12:35:09.368321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:09.367779    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk" podUID="29c6e230-f0a0-4d1f-bb69-db84edee3f9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 02 12:35:10.375752 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:10.375714    4942 patch_prober.go:29] interesting pod/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 02 12:35:10.376391 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:10.376357    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk" podUID="29c6e230-f0a0-4d1f-bb69-db84edee3f9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 02 12:35:10.723204 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:10.723124    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"]
Nov 02 12:35:10.723367 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:10.723342    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg" podUID="107ae880-4744-43a3-a3cb-b90e46faadb6" containerName="maas-api" containerID="cri-o://aa94dd7d323dc07e2c92f0158c488d388a8d4cc8e271d204196b9853d3dfcba6" gracePeriod=30
Nov 02 12:35:11.012903 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:11.011545    4942 generic.go:356] "Generic (PLEG): container finished" podID="107ae880-4744-43a3-a3cb-b90e46faadb6" containerID="aa94dd7d323dc07e2c92f0158c488d388a8d4cc8e271d204196b9853d3dfcba6" exitCode=0
Nov 02 12:35:11.012903 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:11.011618    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg" event={"ID":"107ae880-4744-43a3-a3cb-b90e46faadb6","Type":"ContainerDied","Data":"aa94dd7d323dc07e2c92f0158c488d388a8d4cc8e271d204196b9853d3dfcba6"}
Nov 02 12:35:11.050958 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:11.050926    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:35:11.122067 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:11.122020    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/107ae880-4744-43a3-a3cb-b90e46faadb6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/107ae880-4744-43a3-a3cb-b90e46faadb6-openshift-service-ca\") pod \"107ae880-4744-43a3-a3cb-b90e46faadb6\" (UID: \"107ae880-4744-43a3-a3cb-b90e46faadb6\") "
Nov 02 12:35:11.122254 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:11.122082    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/107ae880-4744-43a3-a3cb-b90e46faadb6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/107ae880-4744-43a3-a3cb-b90e46faadb6-maas-api-tls\") pod \"107ae880-4744-43a3-a3cb-b90e46faadb6\" (UID: \"107ae880-4744-43a3-a3cb-b90e46faadb6\") "
Nov 02 12:35:11.122254 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:11.122236    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/107ae880-4744-43a3-a3cb-b90e46faadb6-kube-api-access-szvtx\" (UniqueName: \"kubernetes.io/projected/107ae880-4744-43a3-a3cb-b90e46faadb6-kube-api-access-szvtx\") pod \"107ae880-4744-43a3-a3cb-b90e46faadb6\" (UID: \"107ae880-4744-43a3-a3cb-b90e46faadb6\") "
Nov 02 12:35:11.122401 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:11.122269    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/107ae880-4744-43a3-a3cb-b90e46faadb6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/107ae880-4744-43a3-a3cb-b90e46faadb6-metrics-certs\") pod \"107ae880-4744-43a3-a3cb-b90e46faadb6\" (UID: \"107ae880-4744-43a3-a3cb-b90e46faadb6\") "
Nov 02 12:35:11.123468 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:11.123118    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/107ae880-4744-43a3-a3cb-b90e46faadb6-openshift-service-ca" pod "107ae880-4744-43a3-a3cb-b90e46faadb6" (UID: "107ae880-4744-43a3-a3cb-b90e46faadb6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:35:11.125259 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:11.125050    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/107ae880-4744-43a3-a3cb-b90e46faadb6-metrics-certs" pod "107ae880-4744-43a3-a3cb-b90e46faadb6" (UID: "107ae880-4744-43a3-a3cb-b90e46faadb6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:35:11.130364 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:11.126475    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/107ae880-4744-43a3-a3cb-b90e46faadb6-kube-api-access-szvtx" pod "107ae880-4744-43a3-a3cb-b90e46faadb6" (UID: "107ae880-4744-43a3-a3cb-b90e46faadb6"). InnerVolumeSpecName "kube-api-access-szvtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:35:11.130364 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:11.129422    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/107ae880-4744-43a3-a3cb-b90e46faadb6-maas-api-tls" pod "107ae880-4744-43a3-a3cb-b90e46faadb6" (UID: "107ae880-4744-43a3-a3cb-b90e46faadb6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:35:11.223911 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:11.223875    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/107ae880-4744-43a3-a3cb-b90e46faadb6-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:35:11.223911 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:11.223907    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/107ae880-4744-43a3-a3cb-b90e46faadb6-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:35:11.224111 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:11.223922    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szvtx\" (UniqueName: \"kubernetes.io/projected/107ae880-4744-43a3-a3cb-b90e46faadb6-kube-api-access-szvtx\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:35:11.224111 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:11.223934    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/107ae880-4744-43a3-a3cb-b90e46faadb6-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:35:11.368720 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:11.368682    4942 patch_prober.go:29] interesting pod/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Nov 02 12:35:11.369005 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:11.368979    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk" podUID="29c6e230-f0a0-4d1f-bb69-db84edee3f9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Nov 02 12:35:11.583321 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:11.582546    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:35:12.020670 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:12.020628    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"
Nov 02 12:35:12.020670 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:12.020645    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg" event={"ID":"107ae880-4744-43a3-a3cb-b90e46faadb6","Type":"ContainerDied","Data":"c405575835fbee5c337c1aee9ee7695eaeba003839d6afb73aad2623a8806b97"}
Nov 02 12:35:12.021011 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:12.020708    4942 scope.go:122] "RemoveContainer" containerID="aa94dd7d323dc07e2c92f0158c488d388a8d4cc8e271d204196b9853d3dfcba6"
Nov 02 12:35:12.186280 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:12.186244    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"]
Nov 02 12:35:12.235886 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:12.235859    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-257e6d-7df6f844bf-9rjjg"]
Nov 02 12:35:12.370644 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:12.370621    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:35:12.371787 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:12.371765    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:35:13.543722 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:13.543689    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="107ae880-4744-43a3-a3cb-b90e46faadb6" path="/var/lib/kubelet/pods/107ae880-4744-43a3-a3cb-b90e46faadb6/volumes"
Nov 02 12:35:16.048881 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:16.048835    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52"]
Nov 02 12:35:16.097986 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:16.097960    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae6808dd-d8x52"]
Nov 02 12:35:17.544626 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:17.544590    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="751b51bc-7be4-4628-a5e3-978e753d65e3" path="/var/lib/kubelet/pods/751b51bc-7be4-4628-a5e3-978e753d65e3/volumes"
Nov 02 12:35:21.693110 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:21.693074    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"]
Nov 02 12:35:21.693573 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:21.693314    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4" podUID="5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" containerName="istio-proxy" containerID="cri-o://460014b0dc2abb36e6461ea45a2a22bb28be899a2e14f5493ba80328dd869248" gracePeriod=30
Nov 02 12:35:23.642981 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:23.642944    4942 patch_prober.go:29] interesting pod/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 12:35:23.643410 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:23.643011    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4" podUID="5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 12:35:26.983425 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:26.983393    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:35:27.105966 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.105933    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istiod-ca-cert\") pod \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") "
Nov 02 12:35:27.106133 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.106033    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-workload-socket\") pod \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") "
Nov 02 12:35:27.106133 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.106087    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-data\") pod \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") "
Nov 02 12:35:27.106223 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.106140    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-kube-api-access-bqpl6\" (UniqueName: \"kubernetes.io/projected/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-kube-api-access-bqpl6\") pod \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") "
Nov 02 12:35:27.106223 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.106177    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-credential-socket\") pod \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") "
Nov 02 12:35:27.106223 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.106211    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-token\" (UniqueName: \"kubernetes.io/projected/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-token\") pod \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") "
Nov 02 12:35:27.106414 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.106287    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-envoy\") pod \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") "
Nov 02 12:35:27.106414 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.106363    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-workload-socket" pod "5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" (UID: "5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:35:27.106414 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.106371    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-podinfo\") pod \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") "
Nov 02 12:35:27.106414 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.106400    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-workload-certs\") pod \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\" (UID: \"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c\") "
Nov 02 12:35:27.106414 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.106397    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-data" pod "5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" (UID: "5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:35:27.106655 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.106522    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-credential-socket" pod "5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" (UID: "5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:35:27.106712 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.106658    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-workload-certs" pod "5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" (UID: "5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:35:27.106772 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.106743    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istiod-ca-cert" pod "5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" (UID: "5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:35:27.106873 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.106855    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:35:27.106948 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.106879    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:35:27.106948 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.106894    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:35:27.106948 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.106908    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:35:27.106948 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.106922    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:35:27.108494 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.108465    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-envoy" pod "5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" (UID: "5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:35:27.108622 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.108533    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-podinfo" pod "5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" (UID: "5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:35:27.108622 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.108576    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-kube-api-access-bqpl6" pod "5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" (UID: "5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c"). InnerVolumeSpecName "kube-api-access-bqpl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:35:27.108622 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.108574    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-token" pod "5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" (UID: "5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:35:27.118998 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.118971    4942 generic.go:356] "Generic (PLEG): container finished" podID="5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" containerID="460014b0dc2abb36e6461ea45a2a22bb28be899a2e14f5493ba80328dd869248" exitCode=0
Nov 02 12:35:27.119122 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.119035    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"
Nov 02 12:35:27.119122 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.119061    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4" event={"ID":"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c","Type":"ContainerDied","Data":"460014b0dc2abb36e6461ea45a2a22bb28be899a2e14f5493ba80328dd869248"}
Nov 02 12:35:27.119239 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.119127    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4" event={"ID":"5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c","Type":"ContainerDied","Data":"e196fedc56560585c0bde2161fa5ae270a3b31eab02e38aa0977268f2b982033"}
Nov 02 12:35:27.119239 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.119163    4942 scope.go:122] "RemoveContainer" containerID="460014b0dc2abb36e6461ea45a2a22bb28be899a2e14f5493ba80328dd869248"
Nov 02 12:35:27.134621 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.134604    4942 scope.go:122] "RemoveContainer" containerID="b6cb99aa29c097599373493eb0427d013e05c0bc8d1b672387298edb6f716407"
Nov 02 12:35:27.143076 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.143058    4942 scope.go:122] "RemoveContainer" containerID="460014b0dc2abb36e6461ea45a2a22bb28be899a2e14f5493ba80328dd869248"
Nov 02 12:35:27.143351 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:35:27.143332    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"460014b0dc2abb36e6461ea45a2a22bb28be899a2e14f5493ba80328dd869248\": container with ID starting with 460014b0dc2abb36e6461ea45a2a22bb28be899a2e14f5493ba80328dd869248 not found: ID does not exist" containerID="460014b0dc2abb36e6461ea45a2a22bb28be899a2e14f5493ba80328dd869248"
Nov 02 12:35:27.143427 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.143362    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"460014b0dc2abb36e6461ea45a2a22bb28be899a2e14f5493ba80328dd869248"} err="failed to get container status \"460014b0dc2abb36e6461ea45a2a22bb28be899a2e14f5493ba80328dd869248\": rpc error: code = NotFound desc = could not find container \"460014b0dc2abb36e6461ea45a2a22bb28be899a2e14f5493ba80328dd869248\": container with ID starting with 460014b0dc2abb36e6461ea45a2a22bb28be899a2e14f5493ba80328dd869248 not found: ID does not exist"
Nov 02 12:35:27.143427 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.143389    4942 scope.go:122] "RemoveContainer" containerID="b6cb99aa29c097599373493eb0427d013e05c0bc8d1b672387298edb6f716407"
Nov 02 12:35:27.143681 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:35:27.143661    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6cb99aa29c097599373493eb0427d013e05c0bc8d1b672387298edb6f716407\": container with ID starting with b6cb99aa29c097599373493eb0427d013e05c0bc8d1b672387298edb6f716407 not found: ID does not exist" containerID="b6cb99aa29c097599373493eb0427d013e05c0bc8d1b672387298edb6f716407"
Nov 02 12:35:27.143724 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.143687    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6cb99aa29c097599373493eb0427d013e05c0bc8d1b672387298edb6f716407"} err="failed to get container status \"b6cb99aa29c097599373493eb0427d013e05c0bc8d1b672387298edb6f716407\": rpc error: code = NotFound desc = could not find container \"b6cb99aa29c097599373493eb0427d013e05c0bc8d1b672387298edb6f716407\": container with ID starting with b6cb99aa29c097599373493eb0427d013e05c0bc8d1b672387298edb6f716407 not found: ID does not exist"
Nov 02 12:35:27.208414 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.208376    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:35:27.208414 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.208403    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:35:27.208414 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.208415    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:35:27.208711 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.208424    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqpl6\" (UniqueName: \"kubernetes.io/projected/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c-kube-api-access-bqpl6\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:35:27.267631 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.267547    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"]
Nov 02 12:35:27.270791 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.270766    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-257e6d-openshift-default-5dfc7946df-rdft4"]
Nov 02 12:35:27.542416 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:35:27.542339    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" path="/var/lib/kubelet/pods/5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c/volumes"
Nov 02 12:35:30.983195 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:35:30.983157    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:36:00.984923 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:36:00.984836    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:36:13.911115 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:13.911077    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:36:13.919092 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:13.919071    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:36:14.252440 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:14.252362    4942 scope.go:122] "RemoveContainer" containerID="db531b34c21564da6eecd343522aedf9d730246c96d26a8a87192686b7166150"
Nov 02 12:36:30.983370 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:36:30.983330    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:36:45.788454 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:45.788412    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"]
Nov 02 12:36:45.789017 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:45.788630    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh" podUID="6144b552-eded-47c0-97a8-800af7f06e67" containerName="maas-api" containerID="cri-o://5d30562d26c39a56637fe86f1e52a2b5bb4f4fe8bed9dca00e27e0295716b351" gracePeriod=30
Nov 02 12:36:45.866517 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:45.866481    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh" podUID="6144b552-eded-47c0-97a8-800af7f06e67" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.112:8443/health\": dial tcp 10.132.0.112:8443: connect: connection refused"
Nov 02 12:36:46.048793 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.048735    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:36:46.134240 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.134206    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6144b552-eded-47c0-97a8-800af7f06e67-kube-api-access-krzb9\" (UniqueName: \"kubernetes.io/projected/6144b552-eded-47c0-97a8-800af7f06e67-kube-api-access-krzb9\") pod \"6144b552-eded-47c0-97a8-800af7f06e67\" (UID: \"6144b552-eded-47c0-97a8-800af7f06e67\") "
Nov 02 12:36:46.134443 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.134282    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6144b552-eded-47c0-97a8-800af7f06e67-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6144b552-eded-47c0-97a8-800af7f06e67-maas-api-tls\") pod \"6144b552-eded-47c0-97a8-800af7f06e67\" (UID: \"6144b552-eded-47c0-97a8-800af7f06e67\") "
Nov 02 12:36:46.134443 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.134354    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6144b552-eded-47c0-97a8-800af7f06e67-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6144b552-eded-47c0-97a8-800af7f06e67-openshift-service-ca\") pod \"6144b552-eded-47c0-97a8-800af7f06e67\" (UID: \"6144b552-eded-47c0-97a8-800af7f06e67\") "
Nov 02 12:36:46.134619 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.134480    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6144b552-eded-47c0-97a8-800af7f06e67-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6144b552-eded-47c0-97a8-800af7f06e67-metrics-certs\") pod \"6144b552-eded-47c0-97a8-800af7f06e67\" (UID: \"6144b552-eded-47c0-97a8-800af7f06e67\") "
Nov 02 12:36:46.134768 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.134740    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6144b552-eded-47c0-97a8-800af7f06e67-openshift-service-ca" pod "6144b552-eded-47c0-97a8-800af7f06e67" (UID: "6144b552-eded-47c0-97a8-800af7f06e67"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:36:46.134905 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.134883    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6144b552-eded-47c0-97a8-800af7f06e67-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:36:46.136864 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.136835    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6144b552-eded-47c0-97a8-800af7f06e67-metrics-certs" pod "6144b552-eded-47c0-97a8-800af7f06e67" (UID: "6144b552-eded-47c0-97a8-800af7f06e67"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:36:46.136986 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.136970    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6144b552-eded-47c0-97a8-800af7f06e67-kube-api-access-krzb9" pod "6144b552-eded-47c0-97a8-800af7f06e67" (UID: "6144b552-eded-47c0-97a8-800af7f06e67"). InnerVolumeSpecName "kube-api-access-krzb9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:36:46.137034 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.136972    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6144b552-eded-47c0-97a8-800af7f06e67-maas-api-tls" pod "6144b552-eded-47c0-97a8-800af7f06e67" (UID: "6144b552-eded-47c0-97a8-800af7f06e67"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:36:46.235787 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.235753    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6144b552-eded-47c0-97a8-800af7f06e67-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:36:46.235787 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.235782    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6144b552-eded-47c0-97a8-800af7f06e67-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:36:46.235787 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.235794    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krzb9\" (UniqueName: \"kubernetes.io/projected/6144b552-eded-47c0-97a8-800af7f06e67-kube-api-access-krzb9\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:36:46.637423 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.637390    4942 generic.go:356] "Generic (PLEG): container finished" podID="6144b552-eded-47c0-97a8-800af7f06e67" containerID="5d30562d26c39a56637fe86f1e52a2b5bb4f4fe8bed9dca00e27e0295716b351" exitCode=0
Nov 02 12:36:46.637585 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.637436    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"
Nov 02 12:36:46.637585 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.637435    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh" event={"ID":"6144b552-eded-47c0-97a8-800af7f06e67","Type":"ContainerDied","Data":"5d30562d26c39a56637fe86f1e52a2b5bb4f4fe8bed9dca00e27e0295716b351"}
Nov 02 12:36:46.637585 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.637562    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh" event={"ID":"6144b552-eded-47c0-97a8-800af7f06e67","Type":"ContainerDied","Data":"4b61db88eab28987578f7c79aa0d31b83c6730205caf92c3c001e5a320428256"}
Nov 02 12:36:46.637688 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.637591    4942 scope.go:122] "RemoveContainer" containerID="5d30562d26c39a56637fe86f1e52a2b5bb4f4fe8bed9dca00e27e0295716b351"
Nov 02 12:36:46.647903 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.647885    4942 scope.go:122] "RemoveContainer" containerID="5d30562d26c39a56637fe86f1e52a2b5bb4f4fe8bed9dca00e27e0295716b351"
Nov 02 12:36:46.648191 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:36:46.648171    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d30562d26c39a56637fe86f1e52a2b5bb4f4fe8bed9dca00e27e0295716b351\": container with ID starting with 5d30562d26c39a56637fe86f1e52a2b5bb4f4fe8bed9dca00e27e0295716b351 not found: ID does not exist" containerID="5d30562d26c39a56637fe86f1e52a2b5bb4f4fe8bed9dca00e27e0295716b351"
Nov 02 12:36:46.648246 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.648200    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d30562d26c39a56637fe86f1e52a2b5bb4f4fe8bed9dca00e27e0295716b351"} err="failed to get container status \"5d30562d26c39a56637fe86f1e52a2b5bb4f4fe8bed9dca00e27e0295716b351\": rpc error: code = NotFound desc = could not find container \"5d30562d26c39a56637fe86f1e52a2b5bb4f4fe8bed9dca00e27e0295716b351\": container with ID starting with 5d30562d26c39a56637fe86f1e52a2b5bb4f4fe8bed9dca00e27e0295716b351 not found: ID does not exist"
Nov 02 12:36:46.805050 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.805014    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"]
Nov 02 12:36:46.854734 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:46.854706    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-6373b9-77d47bcdf-qf2kh"]
Nov 02 12:36:47.542141 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:47.542105    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6144b552-eded-47c0-97a8-800af7f06e67" path="/var/lib/kubelet/pods/6144b552-eded-47c0-97a8-800af7f06e67/volumes"
Nov 02 12:36:54.385900 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:54.385864    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"]
Nov 02 12:36:54.386327 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:54.386074    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk" podUID="29c6e230-f0a0-4d1f-bb69-db84edee3f9e" containerName="istio-proxy" containerID="cri-o://d98c80a01e7049ba42c410fecb46f3ad6ba15cc7510db119e2581fff1176f93c" gracePeriod=30
Nov 02 12:36:57.373943 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:57.373908    4942 patch_prober.go:29] interesting pod/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 12:36:57.374422 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:57.373990    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk" podUID="29c6e230-f0a0-4d1f-bb69-db84edee3f9e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 12:36:59.642766 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.642740    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:36:59.669224 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.669164    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-kube-api-access-jx8vt\" (UniqueName: \"kubernetes.io/projected/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-kube-api-access-jx8vt\") pod \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") "
Nov 02 12:36:59.669350 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.669284    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-workload-socket\") pod \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") "
Nov 02 12:36:59.669415 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.669362    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-envoy\") pod \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") "
Nov 02 12:36:59.669415 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.669398    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-token\" (UniqueName: \"kubernetes.io/projected/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-token\") pod \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") "
Nov 02 12:36:59.669506 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.669430    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-podinfo\") pod \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") "
Nov 02 12:36:59.669506 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.669488    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istiod-ca-cert\") pod \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") "
Nov 02 12:36:59.669588 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.669518    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-data\") pod \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") "
Nov 02 12:36:59.669588 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.669566    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-workload-certs\") pod \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") "
Nov 02 12:36:59.669698 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.669586    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-workload-socket" pod "29c6e230-f0a0-4d1f-bb69-db84edee3f9e" (UID: "29c6e230-f0a0-4d1f-bb69-db84edee3f9e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:36:59.669698 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.669602    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-credential-socket\") pod \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\" (UID: \"29c6e230-f0a0-4d1f-bb69-db84edee3f9e\") "
Nov 02 12:36:59.669847 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.669812    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-data" pod "29c6e230-f0a0-4d1f-bb69-db84edee3f9e" (UID: "29c6e230-f0a0-4d1f-bb69-db84edee3f9e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:36:59.669934 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.669875    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istiod-ca-cert" pod "29c6e230-f0a0-4d1f-bb69-db84edee3f9e" (UID: "29c6e230-f0a0-4d1f-bb69-db84edee3f9e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:36:59.669934 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.669832    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-workload-certs" pod "29c6e230-f0a0-4d1f-bb69-db84edee3f9e" (UID: "29c6e230-f0a0-4d1f-bb69-db84edee3f9e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:36:59.670043 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.670026    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-credential-socket" pod "29c6e230-f0a0-4d1f-bb69-db84edee3f9e" (UID: "29c6e230-f0a0-4d1f-bb69-db84edee3f9e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:36:59.670095 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.670031    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:36:59.670095 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.670064    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:36:59.670095 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.670074    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:36:59.670095 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.670083    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:36:59.671996 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.671970    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-envoy" pod "29c6e230-f0a0-4d1f-bb69-db84edee3f9e" (UID: "29c6e230-f0a0-4d1f-bb69-db84edee3f9e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:36:59.672103 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.671970    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-podinfo" pod "29c6e230-f0a0-4d1f-bb69-db84edee3f9e" (UID: "29c6e230-f0a0-4d1f-bb69-db84edee3f9e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:36:59.672103 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.672029    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-kube-api-access-jx8vt" pod "29c6e230-f0a0-4d1f-bb69-db84edee3f9e" (UID: "29c6e230-f0a0-4d1f-bb69-db84edee3f9e"). InnerVolumeSpecName "kube-api-access-jx8vt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:36:59.672103 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.672044    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-token" pod "29c6e230-f0a0-4d1f-bb69-db84edee3f9e" (UID: "29c6e230-f0a0-4d1f-bb69-db84edee3f9e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:36:59.731126 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.731095    4942 generic.go:356] "Generic (PLEG): container finished" podID="29c6e230-f0a0-4d1f-bb69-db84edee3f9e" containerID="d98c80a01e7049ba42c410fecb46f3ad6ba15cc7510db119e2581fff1176f93c" exitCode=0
Nov 02 12:36:59.731252 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.731144    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"
Nov 02 12:36:59.731252 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.731175    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk" event={"ID":"29c6e230-f0a0-4d1f-bb69-db84edee3f9e","Type":"ContainerDied","Data":"d98c80a01e7049ba42c410fecb46f3ad6ba15cc7510db119e2581fff1176f93c"}
Nov 02 12:36:59.731252 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.731227    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk" event={"ID":"29c6e230-f0a0-4d1f-bb69-db84edee3f9e","Type":"ContainerDied","Data":"93b1910ae2fca0be70b10fe09e1109a4725618759a151e53fa2026878fab0a29"}
Nov 02 12:36:59.731252 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.731250    4942 scope.go:122] "RemoveContainer" containerID="d98c80a01e7049ba42c410fecb46f3ad6ba15cc7510db119e2581fff1176f93c"
Nov 02 12:36:59.741039 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.741022    4942 scope.go:122] "RemoveContainer" containerID="e1cff1c42c764ac4c9d4afa98fc6f5ac1cd0e51e6dad64e14a2d2a2ae5b10080"
Nov 02 12:36:59.749333 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.749315    4942 scope.go:122] "RemoveContainer" containerID="d98c80a01e7049ba42c410fecb46f3ad6ba15cc7510db119e2581fff1176f93c"
Nov 02 12:36:59.749589 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:36:59.749571    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d98c80a01e7049ba42c410fecb46f3ad6ba15cc7510db119e2581fff1176f93c\": container with ID starting with d98c80a01e7049ba42c410fecb46f3ad6ba15cc7510db119e2581fff1176f93c not found: ID does not exist" containerID="d98c80a01e7049ba42c410fecb46f3ad6ba15cc7510db119e2581fff1176f93c"
Nov 02 12:36:59.749627 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.749599    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d98c80a01e7049ba42c410fecb46f3ad6ba15cc7510db119e2581fff1176f93c"} err="failed to get container status \"d98c80a01e7049ba42c410fecb46f3ad6ba15cc7510db119e2581fff1176f93c\": rpc error: code = NotFound desc = could not find container \"d98c80a01e7049ba42c410fecb46f3ad6ba15cc7510db119e2581fff1176f93c\": container with ID starting with d98c80a01e7049ba42c410fecb46f3ad6ba15cc7510db119e2581fff1176f93c not found: ID does not exist"
Nov 02 12:36:59.749627 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.749617    4942 scope.go:122] "RemoveContainer" containerID="e1cff1c42c764ac4c9d4afa98fc6f5ac1cd0e51e6dad64e14a2d2a2ae5b10080"
Nov 02 12:36:59.749839 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:36:59.749825    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1cff1c42c764ac4c9d4afa98fc6f5ac1cd0e51e6dad64e14a2d2a2ae5b10080\": container with ID starting with e1cff1c42c764ac4c9d4afa98fc6f5ac1cd0e51e6dad64e14a2d2a2ae5b10080 not found: ID does not exist" containerID="e1cff1c42c764ac4c9d4afa98fc6f5ac1cd0e51e6dad64e14a2d2a2ae5b10080"
Nov 02 12:36:59.749878 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.749844    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1cff1c42c764ac4c9d4afa98fc6f5ac1cd0e51e6dad64e14a2d2a2ae5b10080"} err="failed to get container status \"e1cff1c42c764ac4c9d4afa98fc6f5ac1cd0e51e6dad64e14a2d2a2ae5b10080\": rpc error: code = NotFound desc = could not find container \"e1cff1c42c764ac4c9d4afa98fc6f5ac1cd0e51e6dad64e14a2d2a2ae5b10080\": container with ID starting with e1cff1c42c764ac4c9d4afa98fc6f5ac1cd0e51e6dad64e14a2d2a2ae5b10080 not found: ID does not exist"
Nov 02 12:36:59.770591 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.770569    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:36:59.770591 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.770591    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:36:59.770932 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.770601    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:36:59.770932 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.770612    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:36:59.770932 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.770622    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jx8vt\" (UniqueName: \"kubernetes.io/projected/29c6e230-f0a0-4d1f-bb69-db84edee3f9e-kube-api-access-jx8vt\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:36:59.923599 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.923513    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"]
Nov 02 12:36:59.929970 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:36:59.929941    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-6373b9-openshift-default-6b89455cfc-wkqfk"]
Nov 02 12:37:00.984630 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:37:00.984591    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:37:01.543587 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:37:01.543553    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29c6e230-f0a0-4d1f-bb69-db84edee3f9e" path="/var/lib/kubelet/pods/29c6e230-f0a0-4d1f-bb69-db84edee3f9e/volumes"
Nov 02 12:37:30.982329 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:37:30.982224    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:38:00.982913 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:38:00.982879    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:38:10.287019 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:10.286983    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h"]
Nov 02 12:38:15.287389 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:15.287326    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h" podUID="48eaad6f-ce18-4218-a7c6-878ae119505a" containerName="payload-pre-processing" containerID="cri-o://b1998864e4aae4b0d879b2abd0409597897350b40f62ca902a986a9096fdfb5c" gracePeriod=25
Nov 02 12:38:15.671006 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:15.670972    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"]
Nov 02 12:38:15.671221 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:15.671177    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc" podUID="97173ac8-689b-41ea-9e10-f99054fae648" containerName="maas-api" containerID="cri-o://6a78bb58c1a760e93ccc67cabaa1281671c2fd0f0d8c559d5a8781b2bb37603c" gracePeriod=30
Nov 02 12:38:15.946319 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:15.946282    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:38:16.110456 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.110425    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97173ac8-689b-41ea-9e10-f99054fae648-kube-api-access-hc6q4\" (UniqueName: \"kubernetes.io/projected/97173ac8-689b-41ea-9e10-f99054fae648-kube-api-access-hc6q4\") pod \"97173ac8-689b-41ea-9e10-f99054fae648\" (UID: \"97173ac8-689b-41ea-9e10-f99054fae648\") "
Nov 02 12:38:16.110634 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.110516    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/97173ac8-689b-41ea-9e10-f99054fae648-metrics-certs\" (UniqueName: \"kubernetes.io/secret/97173ac8-689b-41ea-9e10-f99054fae648-metrics-certs\") pod \"97173ac8-689b-41ea-9e10-f99054fae648\" (UID: \"97173ac8-689b-41ea-9e10-f99054fae648\") "
Nov 02 12:38:16.110634 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.110579    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97173ac8-689b-41ea-9e10-f99054fae648-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97173ac8-689b-41ea-9e10-f99054fae648-openshift-service-ca\") pod \"97173ac8-689b-41ea-9e10-f99054fae648\" (UID: \"97173ac8-689b-41ea-9e10-f99054fae648\") "
Nov 02 12:38:16.110634 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.110628    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/97173ac8-689b-41ea-9e10-f99054fae648-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97173ac8-689b-41ea-9e10-f99054fae648-maas-api-tls\") pod \"97173ac8-689b-41ea-9e10-f99054fae648\" (UID: \"97173ac8-689b-41ea-9e10-f99054fae648\") "
Nov 02 12:38:16.110976 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.110944    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97173ac8-689b-41ea-9e10-f99054fae648-openshift-service-ca" pod "97173ac8-689b-41ea-9e10-f99054fae648" (UID: "97173ac8-689b-41ea-9e10-f99054fae648"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:38:16.111134 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.111093    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97173ac8-689b-41ea-9e10-f99054fae648-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:16.112798 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.112771    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97173ac8-689b-41ea-9e10-f99054fae648-kube-api-access-hc6q4" pod "97173ac8-689b-41ea-9e10-f99054fae648" (UID: "97173ac8-689b-41ea-9e10-f99054fae648"). InnerVolumeSpecName "kube-api-access-hc6q4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:38:16.113164 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.113131    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97173ac8-689b-41ea-9e10-f99054fae648-maas-api-tls" pod "97173ac8-689b-41ea-9e10-f99054fae648" (UID: "97173ac8-689b-41ea-9e10-f99054fae648"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:38:16.113164 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.113148    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97173ac8-689b-41ea-9e10-f99054fae648-metrics-certs" pod "97173ac8-689b-41ea-9e10-f99054fae648" (UID: "97173ac8-689b-41ea-9e10-f99054fae648"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:38:16.169799 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.169765    4942 generic.go:356] "Generic (PLEG): container finished" podID="97173ac8-689b-41ea-9e10-f99054fae648" containerID="6a78bb58c1a760e93ccc67cabaa1281671c2fd0f0d8c559d5a8781b2bb37603c" exitCode=0
Nov 02 12:38:16.169952 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.169815    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"
Nov 02 12:38:16.169952 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.169850    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc" event={"ID":"97173ac8-689b-41ea-9e10-f99054fae648","Type":"ContainerDied","Data":"6a78bb58c1a760e93ccc67cabaa1281671c2fd0f0d8c559d5a8781b2bb37603c"}
Nov 02 12:38:16.169952 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.169905    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc" event={"ID":"97173ac8-689b-41ea-9e10-f99054fae648","Type":"ContainerDied","Data":"6f42b58436c003d3596783e6db32d497f0745a41681de6ac27705f2473abe354"}
Nov 02 12:38:16.169952 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.169932    4942 scope.go:122] "RemoveContainer" containerID="6a78bb58c1a760e93ccc67cabaa1281671c2fd0f0d8c559d5a8781b2bb37603c"
Nov 02 12:38:16.198387 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.198359    4942 scope.go:122] "RemoveContainer" containerID="6a78bb58c1a760e93ccc67cabaa1281671c2fd0f0d8c559d5a8781b2bb37603c"
Nov 02 12:38:16.198762 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:38:16.198738    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a78bb58c1a760e93ccc67cabaa1281671c2fd0f0d8c559d5a8781b2bb37603c\": container with ID starting with 6a78bb58c1a760e93ccc67cabaa1281671c2fd0f0d8c559d5a8781b2bb37603c not found: ID does not exist" containerID="6a78bb58c1a760e93ccc67cabaa1281671c2fd0f0d8c559d5a8781b2bb37603c"
Nov 02 12:38:16.198827 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.198775    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a78bb58c1a760e93ccc67cabaa1281671c2fd0f0d8c559d5a8781b2bb37603c"} err="failed to get container status \"6a78bb58c1a760e93ccc67cabaa1281671c2fd0f0d8c559d5a8781b2bb37603c\": rpc error: code = NotFound desc = could not find container \"6a78bb58c1a760e93ccc67cabaa1281671c2fd0f0d8c559d5a8781b2bb37603c\": container with ID starting with 6a78bb58c1a760e93ccc67cabaa1281671c2fd0f0d8c559d5a8781b2bb37603c not found: ID does not exist"
Nov 02 12:38:16.211900 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.211875    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97173ac8-689b-41ea-9e10-f99054fae648-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:16.211900 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.211900    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hc6q4\" (UniqueName: \"kubernetes.io/projected/97173ac8-689b-41ea-9e10-f99054fae648-kube-api-access-hc6q4\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:16.212057 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.211910    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97173ac8-689b-41ea-9e10-f99054fae648-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:16.333530 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.333498    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"]
Nov 02 12:38:16.337652 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:16.337623    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-dd087a-55b774fdb4-szrbc"]
Nov 02 12:38:17.543102 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:17.543068    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97173ac8-689b-41ea-9e10-f99054fae648" path="/var/lib/kubelet/pods/97173ac8-689b-41ea-9e10-f99054fae648/volumes"
Nov 02 12:38:18.901962 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:18.901922    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h" podUID="48eaad6f-ce18-4218-a7c6-878ae119505a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 12:38:19.555354 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:19.555324    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"]
Nov 02 12:38:19.555538 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:19.555513    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz" podUID="c221cddd-0f93-4f81-9b40-1d017ad063fe" containerName="istio-proxy" containerID="cri-o://8e88d51026c9072b07b1018d80ea26b7b3493617ea8abbacf19d133498a8b794" gracePeriod=30
Nov 02 12:38:24.818320 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.818276    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:38:24.911922 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.911893    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c221cddd-0f93-4f81-9b40-1d017ad063fe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c221cddd-0f93-4f81-9b40-1d017ad063fe-istiod-ca-cert\") pod \"c221cddd-0f93-4f81-9b40-1d017ad063fe\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") "
Nov 02 12:38:24.912068 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.911931    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-credential-socket\") pod \"c221cddd-0f93-4f81-9b40-1d017ad063fe\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") "
Nov 02 12:38:24.912068 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.911967    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-podinfo\") pod \"c221cddd-0f93-4f81-9b40-1d017ad063fe\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") "
Nov 02 12:38:24.912068 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.912036    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-workload-socket\") pod \"c221cddd-0f93-4f81-9b40-1d017ad063fe\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") "
Nov 02 12:38:24.912217 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.912102    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-data\") pod \"c221cddd-0f93-4f81-9b40-1d017ad063fe\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") "
Nov 02 12:38:24.912217 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.912142    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c221cddd-0f93-4f81-9b40-1d017ad063fe-kube-api-access-656ms\" (UniqueName: \"kubernetes.io/projected/c221cddd-0f93-4f81-9b40-1d017ad063fe-kube-api-access-656ms\") pod \"c221cddd-0f93-4f81-9b40-1d017ad063fe\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") "
Nov 02 12:38:24.912217 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.912193    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-token\" (UniqueName: \"kubernetes.io/projected/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-token\") pod \"c221cddd-0f93-4f81-9b40-1d017ad063fe\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") "
Nov 02 12:38:24.912401 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.912243    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-envoy\") pod \"c221cddd-0f93-4f81-9b40-1d017ad063fe\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") "
Nov 02 12:38:24.912401 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.912281    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-workload-certs\") pod \"c221cddd-0f93-4f81-9b40-1d017ad063fe\" (UID: \"c221cddd-0f93-4f81-9b40-1d017ad063fe\") "
Nov 02 12:38:24.912401 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.912326    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-credential-socket" pod "c221cddd-0f93-4f81-9b40-1d017ad063fe" (UID: "c221cddd-0f93-4f81-9b40-1d017ad063fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:38:24.912401 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.912335    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c221cddd-0f93-4f81-9b40-1d017ad063fe-istiod-ca-cert" pod "c221cddd-0f93-4f81-9b40-1d017ad063fe" (UID: "c221cddd-0f93-4f81-9b40-1d017ad063fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:38:24.912603 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.912431    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-workload-socket" pod "c221cddd-0f93-4f81-9b40-1d017ad063fe" (UID: "c221cddd-0f93-4f81-9b40-1d017ad063fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:38:24.912603 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.912514    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-data" pod "c221cddd-0f93-4f81-9b40-1d017ad063fe" (UID: "c221cddd-0f93-4f81-9b40-1d017ad063fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:38:24.912603 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.912587    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-workload-certs" pod "c221cddd-0f93-4f81-9b40-1d017ad063fe" (UID: "c221cddd-0f93-4f81-9b40-1d017ad063fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:38:24.912890 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.912868    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c221cddd-0f93-4f81-9b40-1d017ad063fe-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:24.912962 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.912896    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:24.912962 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.912906    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:24.912962 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.912919    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:24.912962 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.912932    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:24.914346 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.914319    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-envoy" pod "c221cddd-0f93-4f81-9b40-1d017ad063fe" (UID: "c221cddd-0f93-4f81-9b40-1d017ad063fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:38:24.914485 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.914461    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c221cddd-0f93-4f81-9b40-1d017ad063fe-kube-api-access-656ms" pod "c221cddd-0f93-4f81-9b40-1d017ad063fe" (UID: "c221cddd-0f93-4f81-9b40-1d017ad063fe"). InnerVolumeSpecName "kube-api-access-656ms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:38:24.914649 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.914633    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-podinfo" pod "c221cddd-0f93-4f81-9b40-1d017ad063fe" (UID: "c221cddd-0f93-4f81-9b40-1d017ad063fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:38:24.914715 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:24.914672    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-token" pod "c221cddd-0f93-4f81-9b40-1d017ad063fe" (UID: "c221cddd-0f93-4f81-9b40-1d017ad063fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:38:25.013714 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:25.013677    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:25.013714 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:25.013704    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-656ms\" (UniqueName: \"kubernetes.io/projected/c221cddd-0f93-4f81-9b40-1d017ad063fe-kube-api-access-656ms\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:25.013714 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:25.013715    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:25.013714 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:25.013724    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c221cddd-0f93-4f81-9b40-1d017ad063fe-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:25.223788 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:25.223708    4942 generic.go:356] "Generic (PLEG): container finished" podID="c221cddd-0f93-4f81-9b40-1d017ad063fe" containerID="8e88d51026c9072b07b1018d80ea26b7b3493617ea8abbacf19d133498a8b794" exitCode=0
Nov 02 12:38:25.223788 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:25.223754    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"
Nov 02 12:38:25.223960 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:25.223785    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz" event={"ID":"c221cddd-0f93-4f81-9b40-1d017ad063fe","Type":"ContainerDied","Data":"8e88d51026c9072b07b1018d80ea26b7b3493617ea8abbacf19d133498a8b794"}
Nov 02 12:38:25.223960 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:25.223839    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz" event={"ID":"c221cddd-0f93-4f81-9b40-1d017ad063fe","Type":"ContainerDied","Data":"c69df4c20b6d2ad8afe5bf2ac1dad93d1a7d7966aa11b0a2bd4d2e00592adb02"}
Nov 02 12:38:25.223960 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:25.223863    4942 scope.go:122] "RemoveContainer" containerID="8e88d51026c9072b07b1018d80ea26b7b3493617ea8abbacf19d133498a8b794"
Nov 02 12:38:25.234167 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:25.234151    4942 scope.go:122] "RemoveContainer" containerID="6cfae5db3ae651812b857994c0b949e0f2232b1597102c5cc7c9af5a79fdcf5d"
Nov 02 12:38:25.242458 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:25.242441    4942 scope.go:122] "RemoveContainer" containerID="8e88d51026c9072b07b1018d80ea26b7b3493617ea8abbacf19d133498a8b794"
Nov 02 12:38:25.242718 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:38:25.242697    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e88d51026c9072b07b1018d80ea26b7b3493617ea8abbacf19d133498a8b794\": container with ID starting with 8e88d51026c9072b07b1018d80ea26b7b3493617ea8abbacf19d133498a8b794 not found: ID does not exist" containerID="8e88d51026c9072b07b1018d80ea26b7b3493617ea8abbacf19d133498a8b794"
Nov 02 12:38:25.242775 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:25.242726    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e88d51026c9072b07b1018d80ea26b7b3493617ea8abbacf19d133498a8b794"} err="failed to get container status \"8e88d51026c9072b07b1018d80ea26b7b3493617ea8abbacf19d133498a8b794\": rpc error: code = NotFound desc = could not find container \"8e88d51026c9072b07b1018d80ea26b7b3493617ea8abbacf19d133498a8b794\": container with ID starting with 8e88d51026c9072b07b1018d80ea26b7b3493617ea8abbacf19d133498a8b794 not found: ID does not exist"
Nov 02 12:38:25.242775 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:25.242745    4942 scope.go:122] "RemoveContainer" containerID="6cfae5db3ae651812b857994c0b949e0f2232b1597102c5cc7c9af5a79fdcf5d"
Nov 02 12:38:25.242998 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:38:25.242968    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cfae5db3ae651812b857994c0b949e0f2232b1597102c5cc7c9af5a79fdcf5d\": container with ID starting with 6cfae5db3ae651812b857994c0b949e0f2232b1597102c5cc7c9af5a79fdcf5d not found: ID does not exist" containerID="6cfae5db3ae651812b857994c0b949e0f2232b1597102c5cc7c9af5a79fdcf5d"
Nov 02 12:38:25.243055 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:25.242996    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cfae5db3ae651812b857994c0b949e0f2232b1597102c5cc7c9af5a79fdcf5d"} err="failed to get container status \"6cfae5db3ae651812b857994c0b949e0f2232b1597102c5cc7c9af5a79fdcf5d\": rpc error: code = NotFound desc = could not find container \"6cfae5db3ae651812b857994c0b949e0f2232b1597102c5cc7c9af5a79fdcf5d\": container with ID starting with 6cfae5db3ae651812b857994c0b949e0f2232b1597102c5cc7c9af5a79fdcf5d not found: ID does not exist"
Nov 02 12:38:25.387599 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:25.387567    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"]
Nov 02 12:38:25.391683 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:25.391657    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-dd087a-openshift-default-5b6fb74d8c-4hjvz"]
Nov 02 12:38:25.542530 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:25.542464    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c221cddd-0f93-4f81-9b40-1d017ad063fe" path="/var/lib/kubelet/pods/c221cddd-0f93-4f81-9b40-1d017ad063fe/volumes"
Nov 02 12:38:27.091778 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:27.091744    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"]
Nov 02 12:38:27.092171 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:27.091965    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt" podUID="21dddcb9-f9ca-4fae-9744-1d03cbeb529d" containerName="maas-api" containerID="cri-o://2ac721eb8c3152e6acd340db8c9590220c16988bda2414fcfb6091dc411d8f2a" gracePeriod=30
Nov 02 12:38:27.240723 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:27.240693    4942 generic.go:356] "Generic (PLEG): container finished" podID="21dddcb9-f9ca-4fae-9744-1d03cbeb529d" containerID="2ac721eb8c3152e6acd340db8c9590220c16988bda2414fcfb6091dc411d8f2a" exitCode=0
Nov 02 12:38:27.240897 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:27.240752    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt" event={"ID":"21dddcb9-f9ca-4fae-9744-1d03cbeb529d","Type":"ContainerDied","Data":"2ac721eb8c3152e6acd340db8c9590220c16988bda2414fcfb6091dc411d8f2a"}
Nov 02 12:38:27.371901 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:27.371876    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:38:27.437567 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:27.437531    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-openshift-service-ca\") pod \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\" (UID: \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\") "
Nov 02 12:38:27.437734 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:27.437606    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-maas-api-tls\") pod \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\" (UID: \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\") "
Nov 02 12:38:27.437734 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:27.437647    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-metrics-certs\") pod \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\" (UID: \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\") "
Nov 02 12:38:27.437734 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:27.437688    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-kube-api-access-xq762\" (UniqueName: \"kubernetes.io/projected/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-kube-api-access-xq762\") pod \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\" (UID: \"21dddcb9-f9ca-4fae-9744-1d03cbeb529d\") "
Nov 02 12:38:27.437958 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:27.437935    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-openshift-service-ca" pod "21dddcb9-f9ca-4fae-9744-1d03cbeb529d" (UID: "21dddcb9-f9ca-4fae-9744-1d03cbeb529d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:38:27.438126 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:27.438106    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:27.439797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:27.439761    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-metrics-certs" pod "21dddcb9-f9ca-4fae-9744-1d03cbeb529d" (UID: "21dddcb9-f9ca-4fae-9744-1d03cbeb529d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:38:27.439899 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:27.439828    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-maas-api-tls" pod "21dddcb9-f9ca-4fae-9744-1d03cbeb529d" (UID: "21dddcb9-f9ca-4fae-9744-1d03cbeb529d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 12:38:27.439899 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:27.439832    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-kube-api-access-xq762" pod "21dddcb9-f9ca-4fae-9744-1d03cbeb529d" (UID: "21dddcb9-f9ca-4fae-9744-1d03cbeb529d"). InnerVolumeSpecName "kube-api-access-xq762". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:38:27.545177 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:27.541921    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:27.545177 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:27.541956    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq762\" (UniqueName: \"kubernetes.io/projected/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-kube-api-access-xq762\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:27.545177 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:27.541969    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21dddcb9-f9ca-4fae-9744-1d03cbeb529d-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:27.647489 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:27.647453    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png"]
Nov 02 12:38:28.248880 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:28.248841    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"
Nov 02 12:38:28.249405 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:28.248871    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt" event={"ID":"21dddcb9-f9ca-4fae-9744-1d03cbeb529d","Type":"ContainerDied","Data":"21cb87d2b60174150523a88bfdbe2922569c0dcb7e9c153a9a2a549c2c80f008"}
Nov 02 12:38:28.249405 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:28.248945    4942 scope.go:122] "RemoveContainer" containerID="2ac721eb8c3152e6acd340db8c9590220c16988bda2414fcfb6091dc411d8f2a"
Nov 02 12:38:28.413175 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:28.413146    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"]
Nov 02 12:38:28.417940 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:28.417916    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-03c4e8-664b6d4ff4-n8rgt"]
Nov 02 12:38:28.902524 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:28.902488    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h" podUID="48eaad6f-ce18-4218-a7c6-878ae119505a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 12:38:29.542519 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:29.542487    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21dddcb9-f9ca-4fae-9744-1d03cbeb529d" path="/var/lib/kubelet/pods/21dddcb9-f9ca-4fae-9744-1d03cbeb529d/volumes"
Nov 02 12:38:30.983799 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:38:30.983764    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:38:32.647932 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:32.647882    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png" podUID="32bcdc82-f897-4744-a140-f96f352a3422" containerName="payload-pre-processing" containerID="cri-o://48144bce98a2eb6f3918ade653c0a794ceda0a52b27eac34dc359ad18fecc385" gracePeriod=25
Nov 02 12:38:34.781707 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:34.781674    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"]
Nov 02 12:38:34.782111 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:34.781917    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj" podUID="76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" containerName="istio-proxy" containerID="cri-o://1061706a3d4f9ab55aa4630c9f7cefd585fa4d9d028980128888f517c82f3147" gracePeriod=30
Nov 02 12:38:35.549882 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:35.549815    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h"
Nov 02 12:38:35.619755 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:35.619728    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48eaad6f-ce18-4218-a7c6-878ae119505a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48eaad6f-ce18-4218-a7c6-878ae119505a-plugins-config-volume\") pod \"48eaad6f-ce18-4218-a7c6-878ae119505a\" (UID: \"48eaad6f-ce18-4218-a7c6-878ae119505a\") "
Nov 02 12:38:35.619902 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:35.619879    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48eaad6f-ce18-4218-a7c6-878ae119505a-kube-api-access-sc6f2\" (UniqueName: \"kubernetes.io/projected/48eaad6f-ce18-4218-a7c6-878ae119505a-kube-api-access-sc6f2\") pod \"48eaad6f-ce18-4218-a7c6-878ae119505a\" (UID: \"48eaad6f-ce18-4218-a7c6-878ae119505a\") "
Nov 02 12:38:35.620087 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:35.620063    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48eaad6f-ce18-4218-a7c6-878ae119505a-plugins-config-volume" pod "48eaad6f-ce18-4218-a7c6-878ae119505a" (UID: "48eaad6f-ce18-4218-a7c6-878ae119505a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:38:35.620259 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:35.620232    4942 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48eaad6f-ce18-4218-a7c6-878ae119505a-plugins-config-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:35.621994 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:35.621956    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48eaad6f-ce18-4218-a7c6-878ae119505a-kube-api-access-sc6f2" pod "48eaad6f-ce18-4218-a7c6-878ae119505a" (UID: "48eaad6f-ce18-4218-a7c6-878ae119505a"). InnerVolumeSpecName "kube-api-access-sc6f2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:38:35.720912 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:35.720879    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sc6f2\" (UniqueName: \"kubernetes.io/projected/48eaad6f-ce18-4218-a7c6-878ae119505a-kube-api-access-sc6f2\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:36.298921 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:36.298888    4942 generic.go:356] "Generic (PLEG): container finished" podID="48eaad6f-ce18-4218-a7c6-878ae119505a" containerID="b1998864e4aae4b0d879b2abd0409597897350b40f62ca902a986a9096fdfb5c" exitCode=0
Nov 02 12:38:36.299434 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:36.298966    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h" event={"ID":"48eaad6f-ce18-4218-a7c6-878ae119505a","Type":"ContainerDied","Data":"b1998864e4aae4b0d879b2abd0409597897350b40f62ca902a986a9096fdfb5c"}
Nov 02 12:38:36.299434 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:36.299006    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h" event={"ID":"48eaad6f-ce18-4218-a7c6-878ae119505a","Type":"ContainerDied","Data":"d31816409b33445ea1f4f17603d314cc7f59687224e80e81d1739fc9d732a864"}
Nov 02 12:38:36.299434 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:36.299011    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h"
Nov 02 12:38:36.299434 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:36.299026    4942 scope.go:122] "RemoveContainer" containerID="b1998864e4aae4b0d879b2abd0409597897350b40f62ca902a986a9096fdfb5c"
Nov 02 12:38:36.311934 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:36.311912    4942 scope.go:122] "RemoveContainer" containerID="b1998864e4aae4b0d879b2abd0409597897350b40f62ca902a986a9096fdfb5c"
Nov 02 12:38:36.312214 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:38:36.312191    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1998864e4aae4b0d879b2abd0409597897350b40f62ca902a986a9096fdfb5c\": container with ID starting with b1998864e4aae4b0d879b2abd0409597897350b40f62ca902a986a9096fdfb5c not found: ID does not exist" containerID="b1998864e4aae4b0d879b2abd0409597897350b40f62ca902a986a9096fdfb5c"
Nov 02 12:38:36.312342 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:36.312226    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1998864e4aae4b0d879b2abd0409597897350b40f62ca902a986a9096fdfb5c"} err="failed to get container status \"b1998864e4aae4b0d879b2abd0409597897350b40f62ca902a986a9096fdfb5c\": rpc error: code = NotFound desc = could not find container \"b1998864e4aae4b0d879b2abd0409597897350b40f62ca902a986a9096fdfb5c\": container with ID starting with b1998864e4aae4b0d879b2abd0409597897350b40f62ca902a986a9096fdfb5c not found: ID does not exist"
Nov 02 12:38:36.463693 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:36.463664    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h"]
Nov 02 12:38:36.512852 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:36.512812    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-dd087a-5d8c4cc7b7-mhg4h"]
Nov 02 12:38:37.373043 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:37.373008    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png" podUID="32bcdc82-f897-4744-a140-f96f352a3422" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 12:38:37.543211 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:37.543178    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48eaad6f-ce18-4218-a7c6-878ae119505a" path="/var/lib/kubelet/pods/48eaad6f-ce18-4218-a7c6-878ae119505a/volumes"
Nov 02 12:38:39.100633 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:39.100590    4942 patch_prober.go:29] interesting pod/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 12:38:39.101042 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:39.100676    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj" podUID="76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 12:38:40.039940 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.039913    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:38:40.163959 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.163932    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-workload-socket\") pod \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") "
Nov 02 12:38:40.164442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.163982    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-kube-api-access-ldcn9\" (UniqueName: \"kubernetes.io/projected/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-kube-api-access-ldcn9\") pod \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") "
Nov 02 12:38:40.164442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.164026    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-workload-certs\") pod \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") "
Nov 02 12:38:40.164442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.164056    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-token\" (UniqueName: \"kubernetes.io/projected/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-token\") pod \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") "
Nov 02 12:38:40.164442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.164158    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-data\") pod \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") "
Nov 02 12:38:40.164442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.164189    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istiod-ca-cert\") pod \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") "
Nov 02 12:38:40.164442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.164192    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-workload-socket" pod "76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" (UID: "76a878ec-9ca9-47e9-bf4c-8cf19678a4ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:38:40.164442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.164235    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-envoy\") pod \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") "
Nov 02 12:38:40.164442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.164272    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-credential-socket\") pod \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") "
Nov 02 12:38:40.164442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.164343    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-workload-certs" pod "76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" (UID: "76a878ec-9ca9-47e9-bf4c-8cf19678a4ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:38:40.164442 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.164360    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-podinfo\") pod \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\" (UID: \"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee\") "
Nov 02 12:38:40.164883 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.164449    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-data" pod "76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" (UID: "76a878ec-9ca9-47e9-bf4c-8cf19678a4ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:38:40.164883 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.164586    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-credential-socket" pod "76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" (UID: "76a878ec-9ca9-47e9-bf4c-8cf19678a4ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:38:40.164883 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.164608    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istiod-ca-cert" pod "76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" (UID: "76a878ec-9ca9-47e9-bf4c-8cf19678a4ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:38:40.164883 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.164755    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:40.164883 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.164776    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:40.164883 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.164786    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:40.164883 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.164799    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:40.164883 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.164811    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:40.166459 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.166434    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-envoy" pod "76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" (UID: "76a878ec-9ca9-47e9-bf4c-8cf19678a4ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:38:40.166573 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.166495    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-podinfo" pod "76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" (UID: "76a878ec-9ca9-47e9-bf4c-8cf19678a4ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:38:40.166741 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.166721    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-token" pod "76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" (UID: "76a878ec-9ca9-47e9-bf4c-8cf19678a4ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:38:40.166804 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.166722    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-kube-api-access-ldcn9" pod "76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" (UID: "76a878ec-9ca9-47e9-bf4c-8cf19678a4ee"). InnerVolumeSpecName "kube-api-access-ldcn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:38:40.265789 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.265748    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldcn9\" (UniqueName: \"kubernetes.io/projected/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-kube-api-access-ldcn9\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:40.265789 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.265785    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:40.266027 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.265802    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:40.266027 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.265815    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:40.323535 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.323498    4942 generic.go:356] "Generic (PLEG): container finished" podID="76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" containerID="1061706a3d4f9ab55aa4630c9f7cefd585fa4d9d028980128888f517c82f3147" exitCode=0
Nov 02 12:38:40.323700 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.323549    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"
Nov 02 12:38:40.323700 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.323580    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj" event={"ID":"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee","Type":"ContainerDied","Data":"1061706a3d4f9ab55aa4630c9f7cefd585fa4d9d028980128888f517c82f3147"}
Nov 02 12:38:40.323700 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.323656    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj" event={"ID":"76a878ec-9ca9-47e9-bf4c-8cf19678a4ee","Type":"ContainerDied","Data":"30f0bc6ac48d157db9197cc39678d80a66aedd19ad82fd65c2bd065b832be125"}
Nov 02 12:38:40.323700 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.323679    4942 scope.go:122] "RemoveContainer" containerID="1061706a3d4f9ab55aa4630c9f7cefd585fa4d9d028980128888f517c82f3147"
Nov 02 12:38:40.333940 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.333922    4942 scope.go:122] "RemoveContainer" containerID="1061706a3d4f9ab55aa4630c9f7cefd585fa4d9d028980128888f517c82f3147"
Nov 02 12:38:40.334177 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:38:40.334156    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1061706a3d4f9ab55aa4630c9f7cefd585fa4d9d028980128888f517c82f3147\": container with ID starting with 1061706a3d4f9ab55aa4630c9f7cefd585fa4d9d028980128888f517c82f3147 not found: ID does not exist" containerID="1061706a3d4f9ab55aa4630c9f7cefd585fa4d9d028980128888f517c82f3147"
Nov 02 12:38:40.334247 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.334188    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1061706a3d4f9ab55aa4630c9f7cefd585fa4d9d028980128888f517c82f3147"} err="failed to get container status \"1061706a3d4f9ab55aa4630c9f7cefd585fa4d9d028980128888f517c82f3147\": rpc error: code = NotFound desc = could not find container \"1061706a3d4f9ab55aa4630c9f7cefd585fa4d9d028980128888f517c82f3147\": container with ID starting with 1061706a3d4f9ab55aa4630c9f7cefd585fa4d9d028980128888f517c82f3147 not found: ID does not exist"
Nov 02 12:38:40.488691 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.488604    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"]
Nov 02 12:38:40.493126 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:40.493101    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-03c4e8-openshift-default-6f858d7b75-4zsnj"]
Nov 02 12:38:41.542046 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:41.542011    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" path="/var/lib/kubelet/pods/76a878ec-9ca9-47e9-bf4c-8cf19678a4ee/volumes"
Nov 02 12:38:47.373826 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:47.373779    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png" podUID="32bcdc82-f897-4744-a140-f96f352a3422" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 12:38:52.906033 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:52.905976    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png"
Nov 02 12:38:52.979798 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:52.979771    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32bcdc82-f897-4744-a140-f96f352a3422-kube-api-access-r2z4c\" (UniqueName: \"kubernetes.io/projected/32bcdc82-f897-4744-a140-f96f352a3422-kube-api-access-r2z4c\") pod \"32bcdc82-f897-4744-a140-f96f352a3422\" (UID: \"32bcdc82-f897-4744-a140-f96f352a3422\") "
Nov 02 12:38:52.979945 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:52.979891    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32bcdc82-f897-4744-a140-f96f352a3422-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32bcdc82-f897-4744-a140-f96f352a3422-plugins-config-volume\") pod \"32bcdc82-f897-4744-a140-f96f352a3422\" (UID: \"32bcdc82-f897-4744-a140-f96f352a3422\") "
Nov 02 12:38:52.980255 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:52.980226    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32bcdc82-f897-4744-a140-f96f352a3422-plugins-config-volume" pod "32bcdc82-f897-4744-a140-f96f352a3422" (UID: "32bcdc82-f897-4744-a140-f96f352a3422"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:38:52.981804 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:52.981783    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32bcdc82-f897-4744-a140-f96f352a3422-kube-api-access-r2z4c" pod "32bcdc82-f897-4744-a140-f96f352a3422" (UID: "32bcdc82-f897-4744-a140-f96f352a3422"). InnerVolumeSpecName "kube-api-access-r2z4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:38:53.080997 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:53.080964    4942 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32bcdc82-f897-4744-a140-f96f352a3422-plugins-config-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:53.080997 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:53.080992    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2z4c\" (UniqueName: \"kubernetes.io/projected/32bcdc82-f897-4744-a140-f96f352a3422-kube-api-access-r2z4c\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:38:53.407956 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:53.407918    4942 generic.go:356] "Generic (PLEG): container finished" podID="32bcdc82-f897-4744-a140-f96f352a3422" containerID="48144bce98a2eb6f3918ade653c0a794ceda0a52b27eac34dc359ad18fecc385" exitCode=0
Nov 02 12:38:53.408162 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:53.407983    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png"
Nov 02 12:38:53.408162 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:53.408004    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png" event={"ID":"32bcdc82-f897-4744-a140-f96f352a3422","Type":"ContainerDied","Data":"48144bce98a2eb6f3918ade653c0a794ceda0a52b27eac34dc359ad18fecc385"}
Nov 02 12:38:53.408162 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:53.408070    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png" event={"ID":"32bcdc82-f897-4744-a140-f96f352a3422","Type":"ContainerDied","Data":"b7880bcea0ad5f0a399ee672c137b267115e535de0c4f1ab3e436d246503a248"}
Nov 02 12:38:53.408162 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:53.408102    4942 scope.go:122] "RemoveContainer" containerID="48144bce98a2eb6f3918ade653c0a794ceda0a52b27eac34dc359ad18fecc385"
Nov 02 12:38:53.417139 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:53.417121    4942 scope.go:122] "RemoveContainer" containerID="48144bce98a2eb6f3918ade653c0a794ceda0a52b27eac34dc359ad18fecc385"
Nov 02 12:38:53.417414 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:38:53.417395    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48144bce98a2eb6f3918ade653c0a794ceda0a52b27eac34dc359ad18fecc385\": container with ID starting with 48144bce98a2eb6f3918ade653c0a794ceda0a52b27eac34dc359ad18fecc385 not found: ID does not exist" containerID="48144bce98a2eb6f3918ade653c0a794ceda0a52b27eac34dc359ad18fecc385"
Nov 02 12:38:53.417479 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:53.417423    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48144bce98a2eb6f3918ade653c0a794ceda0a52b27eac34dc359ad18fecc385"} err="failed to get container status \"48144bce98a2eb6f3918ade653c0a794ceda0a52b27eac34dc359ad18fecc385\": rpc error: code = NotFound desc = could not find container \"48144bce98a2eb6f3918ade653c0a794ceda0a52b27eac34dc359ad18fecc385\": container with ID starting with 48144bce98a2eb6f3918ade653c0a794ceda0a52b27eac34dc359ad18fecc385 not found: ID does not exist"
Nov 02 12:38:53.574040 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:53.574007    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png"]
Nov 02 12:38:53.623521 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:53.623488    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-03c4e8-7df6c4df79-t5png"]
Nov 02 12:38:55.542739 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:38:55.542704    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32bcdc82-f897-4744-a140-f96f352a3422" path="/var/lib/kubelet/pods/32bcdc82-f897-4744-a140-f96f352a3422/volumes"
Nov 02 12:39:00.983099 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:39:00.983059    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:39:30.983485 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:39:30.983446    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:39:42.908701 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:42.908662    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"]
Nov 02 12:39:42.909229 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:42.908933    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd" podUID="d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" containerName="istio-proxy" containerID="cri-o://46db96d06b0bae9101aa712bb4b40b7cd408432cb5efd47b9eb936f957a3b7da" gracePeriod=30
Nov 02 12:39:46.087031 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:46.086991    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-jmqgd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 12:39:46.087664 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:46.087060    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd" podUID="d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 12:39:48.177909 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.177885    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:39:48.227933 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.227854    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istiod-ca-cert\") pod \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") "
Nov 02 12:39:48.227933 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.227901    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-data\") pod \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") "
Nov 02 12:39:48.227933 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.227923    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-workload-socket\") pod \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") "
Nov 02 12:39:48.228216 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.227962    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-podinfo\") pod \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") "
Nov 02 12:39:48.228216 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.227995    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-envoy\") pod \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") "
Nov 02 12:39:48.228216 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.228030    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-token\" (UniqueName: \"kubernetes.io/projected/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-token\") pod \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") "
Nov 02 12:39:48.228216 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.228120    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-credential-socket\") pod \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") "
Nov 02 12:39:48.228216 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.228198    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-workload-certs\") pod \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") "
Nov 02 12:39:48.228539 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.228234    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-kube-api-access-bn58f\" (UniqueName: \"kubernetes.io/projected/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-kube-api-access-bn58f\") pod \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\" (UID: \"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a\") "
Nov 02 12:39:48.228539 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.228238    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-data" pod "d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" (UID: "d2e3be20-09b2-4bfc-8865-11cbb68d8e2a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:39:48.228539 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.228290    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-workload-socket" pod "d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" (UID: "d2e3be20-09b2-4bfc-8865-11cbb68d8e2a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:39:48.228539 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.228349    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istiod-ca-cert" pod "d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" (UID: "d2e3be20-09b2-4bfc-8865-11cbb68d8e2a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:39:48.228539 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.228512    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-workload-certs" pod "d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" (UID: "d2e3be20-09b2-4bfc-8865-11cbb68d8e2a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:39:48.228791 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.228653    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:39:48.228791 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.228674    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:39:48.228791 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.228676    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-credential-socket" pod "d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" (UID: "d2e3be20-09b2-4bfc-8865-11cbb68d8e2a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:39:48.228791 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.228688    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:39:48.228791 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.228736    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:39:48.230223 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.230203    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-podinfo" pod "d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" (UID: "d2e3be20-09b2-4bfc-8865-11cbb68d8e2a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:39:48.230372 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.230353    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-envoy" pod "d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" (UID: "d2e3be20-09b2-4bfc-8865-11cbb68d8e2a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:39:48.230821 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.230796    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-kube-api-access-bn58f" pod "d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" (UID: "d2e3be20-09b2-4bfc-8865-11cbb68d8e2a"). InnerVolumeSpecName "kube-api-access-bn58f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:39:48.230821 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.230800    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-token" pod "d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" (UID: "d2e3be20-09b2-4bfc-8865-11cbb68d8e2a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:39:48.329797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.329757    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:39:48.329797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.329786    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:39:48.329797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.329796    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:39:48.329797 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.329805    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:39:48.330157 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.329814    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bn58f\" (UniqueName: \"kubernetes.io/projected/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a-kube-api-access-bn58f\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:39:48.714047 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.714015    4942 generic.go:356] "Generic (PLEG): container finished" podID="d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" containerID="46db96d06b0bae9101aa712bb4b40b7cd408432cb5efd47b9eb936f957a3b7da" exitCode=0
Nov 02 12:39:48.714225 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.714062    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"
Nov 02 12:39:48.714225 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.714084    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd" event={"ID":"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a","Type":"ContainerDied","Data":"46db96d06b0bae9101aa712bb4b40b7cd408432cb5efd47b9eb936f957a3b7da"}
Nov 02 12:39:48.714225 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.714128    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd" event={"ID":"d2e3be20-09b2-4bfc-8865-11cbb68d8e2a","Type":"ContainerDied","Data":"4f82ba09b8a31184c4939304d13637c0f9a30a3333bc93bb5f4d9a7dc0b6af56"}
Nov 02 12:39:48.714225 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.714149    4942 scope.go:122] "RemoveContainer" containerID="46db96d06b0bae9101aa712bb4b40b7cd408432cb5efd47b9eb936f957a3b7da"
Nov 02 12:39:48.726205 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.726185    4942 scope.go:122] "RemoveContainer" containerID="46db96d06b0bae9101aa712bb4b40b7cd408432cb5efd47b9eb936f957a3b7da"
Nov 02 12:39:48.726524 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:39:48.726499    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46db96d06b0bae9101aa712bb4b40b7cd408432cb5efd47b9eb936f957a3b7da\": container with ID starting with 46db96d06b0bae9101aa712bb4b40b7cd408432cb5efd47b9eb936f957a3b7da not found: ID does not exist" containerID="46db96d06b0bae9101aa712bb4b40b7cd408432cb5efd47b9eb936f957a3b7da"
Nov 02 12:39:48.726595 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.726536    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46db96d06b0bae9101aa712bb4b40b7cd408432cb5efd47b9eb936f957a3b7da"} err="failed to get container status \"46db96d06b0bae9101aa712bb4b40b7cd408432cb5efd47b9eb936f957a3b7da\": rpc error: code = NotFound desc = could not find container \"46db96d06b0bae9101aa712bb4b40b7cd408432cb5efd47b9eb936f957a3b7da\": container with ID starting with 46db96d06b0bae9101aa712bb4b40b7cd408432cb5efd47b9eb936f957a3b7da not found: ID does not exist"
Nov 02 12:39:48.880225 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.880191    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"]
Nov 02 12:39:48.883407 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:48.883381    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-jmqgd"]
Nov 02 12:39:49.542103 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:39:49.542065    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" path="/var/lib/kubelet/pods/d2e3be20-09b2-4bfc-8865-11cbb68d8e2a/volumes"
Nov 02 12:40:00.983439 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:40:00.983402    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:40:30.982509 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:40:30.982431    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:40:43.381273 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:43.381237    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"]
Nov 02 12:40:43.381700 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:43.381462    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f" podUID="e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" containerName="istio-proxy" containerID="cri-o://c244b9bbd4f49229f0e9d85ffc4ee3fd4e6445e1b1846a35437fe0b3d516ebdb" gracePeriod=30
Nov 02 12:40:45.711754 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:45.711716    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-62r9f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 12:40:45.712216 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:45.711775    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f" podUID="e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 12:40:48.642549 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.642524    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:40:48.731952 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.731861    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-podinfo\") pod \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") "
Nov 02 12:40:48.731952 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.731916    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-credential-socket\") pod \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") "
Nov 02 12:40:48.731952 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.731943    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istiod-ca-cert\") pod \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") "
Nov 02 12:40:48.732252 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.731978    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-envoy\") pod \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") "
Nov 02 12:40:48.732252 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.732024    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-workload-certs\") pod \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") "
Nov 02 12:40:48.732252 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.732055    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-workload-socket\") pod \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") "
Nov 02 12:40:48.732252 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.732105    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-data\") pod \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") "
Nov 02 12:40:48.732252 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.732160    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-token\" (UniqueName: \"kubernetes.io/projected/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-token\") pod \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") "
Nov 02 12:40:48.732546 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.732321    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-credential-socket" pod "e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" (UID: "e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:40:48.732546 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.732444    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-workload-certs" pod "e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" (UID: "e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:40:48.732546 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.732445    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-workload-socket" pod "e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" (UID: "e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:40:48.732546 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.732462    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istiod-ca-cert" pod "e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" (UID: "e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:40:48.732546 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.732465    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-data" pod "e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" (UID: "e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:40:48.732734 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.732568    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-kube-api-access-wb8rs\" (UniqueName: \"kubernetes.io/projected/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-kube-api-access-wb8rs\") pod \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\" (UID: \"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb\") "
Nov 02 12:40:48.733051 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.732992    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:40:48.733051 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.733024    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:40:48.733051 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.733041    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:40:48.733348 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.733056    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:40:48.733348 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.733070    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:40:48.734195 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.734166    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-envoy" pod "e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" (UID: "e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 12:40:48.734365 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.734208    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-podinfo" pod "e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" (UID: "e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 12:40:48.734512 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.734493    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-token" pod "e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" (UID: "e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:40:48.734582 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.734567    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-kube-api-access-wb8rs" pod "e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" (UID: "e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb"). InnerVolumeSpecName "kube-api-access-wb8rs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:40:48.833613 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.833582    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wb8rs\" (UniqueName: \"kubernetes.io/projected/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-kube-api-access-wb8rs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:40:48.833613 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.833610    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:40:48.833613 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.833619    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:40:48.833932 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:48.833628    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:40:49.050044 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:49.049951    4942 generic.go:356] "Generic (PLEG): container finished" podID="e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" containerID="c244b9bbd4f49229f0e9d85ffc4ee3fd4e6445e1b1846a35437fe0b3d516ebdb" exitCode=0
Nov 02 12:40:49.050044 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:49.050006    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"
Nov 02 12:40:49.050044 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:49.050031    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f" event={"ID":"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb","Type":"ContainerDied","Data":"c244b9bbd4f49229f0e9d85ffc4ee3fd4e6445e1b1846a35437fe0b3d516ebdb"}
Nov 02 12:40:49.050340 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:49.050089    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f" event={"ID":"e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb","Type":"ContainerDied","Data":"64e2f4328c8474a3ae74c82bad7bd0063962e0f93d97960f5aece30052fe6c82"}
Nov 02 12:40:49.050340 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:49.050111    4942 scope.go:122] "RemoveContainer" containerID="c244b9bbd4f49229f0e9d85ffc4ee3fd4e6445e1b1846a35437fe0b3d516ebdb"
Nov 02 12:40:49.069825 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:49.069799    4942 scope.go:122] "RemoveContainer" containerID="c244b9bbd4f49229f0e9d85ffc4ee3fd4e6445e1b1846a35437fe0b3d516ebdb"
Nov 02 12:40:49.070391 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:40:49.070339    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c244b9bbd4f49229f0e9d85ffc4ee3fd4e6445e1b1846a35437fe0b3d516ebdb\": container with ID starting with c244b9bbd4f49229f0e9d85ffc4ee3fd4e6445e1b1846a35437fe0b3d516ebdb not found: ID does not exist" containerID="c244b9bbd4f49229f0e9d85ffc4ee3fd4e6445e1b1846a35437fe0b3d516ebdb"
Nov 02 12:40:49.070391 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:49.070370    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c244b9bbd4f49229f0e9d85ffc4ee3fd4e6445e1b1846a35437fe0b3d516ebdb"} err="failed to get container status \"c244b9bbd4f49229f0e9d85ffc4ee3fd4e6445e1b1846a35437fe0b3d516ebdb\": rpc error: code = NotFound desc = could not find container \"c244b9bbd4f49229f0e9d85ffc4ee3fd4e6445e1b1846a35437fe0b3d516ebdb\": container with ID starting with c244b9bbd4f49229f0e9d85ffc4ee3fd4e6445e1b1846a35437fe0b3d516ebdb not found: ID does not exist"
Nov 02 12:40:49.216184 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:49.216143    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"]
Nov 02 12:40:49.222415 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:49.222387    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-62r9f"]
Nov 02 12:40:49.542166 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:40:49.542128    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" path="/var/lib/kubelet/pods/e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb/volumes"
Nov 02 12:41:00.982762 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:41:00.982723    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:41:13.948940 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:41:13.948910    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:41:13.958979 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:41:13.958957    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:41:30.983172 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:41:30.983135    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:42:00.982257 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:42:00.982175    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:42:30.982484 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:42:30.982448    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:43:00.981353 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:43:00.981291    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:43:30.981705 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:43:30.981617    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:44:00.982443 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:44:00.982404    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:44:30.982699 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:44:30.982658    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:45:00.190155 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190078    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp"]
Nov 02 12:45:00.190574 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190485    4942 cpu_manager.go:414] "containerMap: removing container" podUID="5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" containerName="istio-proxy"
Nov 02 12:45:00.190574 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190498    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" containerName="istio-proxy"
Nov 02 12:45:00.190574 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190510    4942 cpu_manager.go:414] "containerMap: removing container" podUID="e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" containerName="istio-proxy"
Nov 02 12:45:00.190574 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190515    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" containerName="istio-proxy"
Nov 02 12:45:00.190574 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190527    4942 cpu_manager.go:414] "containerMap: removing container" podUID="d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" containerName="istio-proxy"
Nov 02 12:45:00.190574 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190531    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" containerName="istio-proxy"
Nov 02 12:45:00.190574 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190541    4942 cpu_manager.go:414] "containerMap: removing container" podUID="29c6e230-f0a0-4d1f-bb69-db84edee3f9e" containerName="istio-proxy"
Nov 02 12:45:00.190574 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190546    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29c6e230-f0a0-4d1f-bb69-db84edee3f9e" containerName="istio-proxy"
Nov 02 12:45:00.190574 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190553    4942 cpu_manager.go:414] "containerMap: removing container" podUID="d61ef5ad-d084-49a0-8241-eb808e8df109" containerName="payload-pre-processing"
Nov 02 12:45:00.190574 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190558    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d61ef5ad-d084-49a0-8241-eb808e8df109" containerName="payload-pre-processing"
Nov 02 12:45:00.190574 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190569    4942 cpu_manager.go:414] "containerMap: removing container" podUID="c221cddd-0f93-4f81-9b40-1d017ad063fe" containerName="istio-proxy"
Nov 02 12:45:00.190574 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190575    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c221cddd-0f93-4f81-9b40-1d017ad063fe" containerName="istio-proxy"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190586    4942 cpu_manager.go:414] "containerMap: removing container" podUID="21dddcb9-f9ca-4fae-9744-1d03cbeb529d" containerName="maas-api"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190591    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21dddcb9-f9ca-4fae-9744-1d03cbeb529d" containerName="maas-api"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190599    4942 cpu_manager.go:414] "containerMap: removing container" podUID="5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" containerName="istio-proxy"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190603    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" containerName="istio-proxy"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190611    4942 cpu_manager.go:414] "containerMap: removing container" podUID="76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" containerName="istio-proxy"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190615    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" containerName="istio-proxy"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190624    4942 cpu_manager.go:414] "containerMap: removing container" podUID="107ae880-4744-43a3-a3cb-b90e46faadb6" containerName="maas-api"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190628    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="107ae880-4744-43a3-a3cb-b90e46faadb6" containerName="maas-api"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190638    4942 cpu_manager.go:414] "containerMap: removing container" podUID="6144b552-eded-47c0-97a8-800af7f06e67" containerName="maas-api"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190642    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6144b552-eded-47c0-97a8-800af7f06e67" containerName="maas-api"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190651    4942 cpu_manager.go:414] "containerMap: removing container" podUID="48eaad6f-ce18-4218-a7c6-878ae119505a" containerName="payload-pre-processing"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190656    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48eaad6f-ce18-4218-a7c6-878ae119505a" containerName="payload-pre-processing"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190662    4942 cpu_manager.go:414] "containerMap: removing container" podUID="32bcdc82-f897-4744-a140-f96f352a3422" containerName="payload-pre-processing"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190666    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32bcdc82-f897-4744-a140-f96f352a3422" containerName="payload-pre-processing"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190674    4942 cpu_manager.go:414] "containerMap: removing container" podUID="97173ac8-689b-41ea-9e10-f99054fae648" containerName="maas-api"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190678    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97173ac8-689b-41ea-9e10-f99054fae648" containerName="maas-api"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190739    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="d61ef5ad-d084-49a0-8241-eb808e8df109" containerName="payload-pre-processing"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190746    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="c221cddd-0f93-4f81-9b40-1d017ad063fe" containerName="istio-proxy"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190753    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="21dddcb9-f9ca-4fae-9744-1d03cbeb529d" containerName="maas-api"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190759    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" containerName="istio-proxy"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190763    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="76a878ec-9ca9-47e9-bf4c-8cf19678a4ee" containerName="istio-proxy"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190769    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="29c6e230-f0a0-4d1f-bb69-db84edee3f9e" containerName="istio-proxy"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190774    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="107ae880-4744-43a3-a3cb-b90e46faadb6" containerName="maas-api"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190780    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="6144b552-eded-47c0-97a8-800af7f06e67" containerName="maas-api"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190786    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="48eaad6f-ce18-4218-a7c6-878ae119505a" containerName="payload-pre-processing"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190790    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="32bcdc82-f897-4744-a140-f96f352a3422" containerName="payload-pre-processing"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190796    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="97173ac8-689b-41ea-9e10-f99054fae648" containerName="maas-api"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190802    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c0ef0fa-78a0-414b-944d-a81c3f3f9e3c" containerName="istio-proxy"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190807    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1ee0c8b-9f5a-4e8c-b468-a0eb7b139ffb" containerName="istio-proxy"
Nov 02 12:45:00.190939 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.190812    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="d2e3be20-09b2-4bfc-8865-11cbb68d8e2a" containerName="istio-proxy"
Nov 02 12:45:00.193854 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.193834    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp"
Nov 02 12:45:00.245799 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.245774    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nbw2z\""
Nov 02 12:45:00.288124 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.288091    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b47x8\" (UniqueName: \"kubernetes.io/projected/9a27f0b8-53b2-4b45-856a-66fa7104e19d-kube-api-access-b47x8\") pod \"maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp\" (UID: \"9a27f0b8-53b2-4b45-856a-66fa7104e19d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp"
Nov 02 12:45:00.288294 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.288231    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a27f0b8-53b2-4b45-856a-66fa7104e19d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp\" (UID: \"9a27f0b8-53b2-4b45-856a-66fa7104e19d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp"
Nov 02 12:45:00.297871 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.297848    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp"]
Nov 02 12:45:00.389144 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.389114    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b47x8\" (UniqueName: \"kubernetes.io/projected/9a27f0b8-53b2-4b45-856a-66fa7104e19d-kube-api-access-b47x8\") pod \"maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp\" (UID: \"9a27f0b8-53b2-4b45-856a-66fa7104e19d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp"
Nov 02 12:45:00.389339 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.389170    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a27f0b8-53b2-4b45-856a-66fa7104e19d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp\" (UID: \"9a27f0b8-53b2-4b45-856a-66fa7104e19d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp"
Nov 02 12:45:00.389778 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.389759    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a27f0b8-53b2-4b45-856a-66fa7104e19d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp\" (UID: \"9a27f0b8-53b2-4b45-856a-66fa7104e19d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp"
Nov 02 12:45:00.452833 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.452780    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b47x8\" (UniqueName: \"kubernetes.io/projected/9a27f0b8-53b2-4b45-856a-66fa7104e19d-kube-api-access-b47x8\") pod \"maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp\" (UID: \"9a27f0b8-53b2-4b45-856a-66fa7104e19d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp"
Nov 02 12:45:00.506581 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.506549    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp"
Nov 02 12:45:00.881324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:00.881285    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp"]
Nov 02 12:45:00.883332 ip-10-0-135-24 kubenswrapper[4942]: W1102 12:45:00.883292    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a27f0b8_53b2_4b45_856a_66fa7104e19d.slice/crio-5c7c7bcc80f054733aed73b109a9adb8db52306a50b246bdda1609015120281c WatchSource:0}: Error finding container 5c7c7bcc80f054733aed73b109a9adb8db52306a50b246bdda1609015120281c: Status 404 returned error can't find the container with id 5c7c7bcc80f054733aed73b109a9adb8db52306a50b246bdda1609015120281c
Nov 02 12:45:00.988452 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:45:00.988420    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:45:01.380832 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:01.380800    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp" event={"ID":"9a27f0b8-53b2-4b45-856a-66fa7104e19d","Type":"ContainerStarted","Data":"5a5594183935f0056eab7bf5280eafad42d59e502e9770b20f103eaeca683465"}
Nov 02 12:45:01.381217 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:01.380867    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp" event={"ID":"9a27f0b8-53b2-4b45-856a-66fa7104e19d","Type":"ContainerStarted","Data":"5c7c7bcc80f054733aed73b109a9adb8db52306a50b246bdda1609015120281c"}
Nov 02 12:45:01.495688 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:01.495638    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp" podStartSLOduration=1.495626351 podStartE2EDuration="1.495626351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 12:45:01.494934013 +0000 UTC m=+2928.066283771" watchObservedRunningTime="2026-11-02 12:45:01.495626351 +0000 UTC m=+2928.066976108"
Nov 02 12:45:02.388286 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:02.388253    4942 generic.go:356] "Generic (PLEG): container finished" podID="9a27f0b8-53b2-4b45-856a-66fa7104e19d" containerID="5a5594183935f0056eab7bf5280eafad42d59e502e9770b20f103eaeca683465" exitCode=60
Nov 02 12:45:02.388902 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:02.388329    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp" event={"ID":"9a27f0b8-53b2-4b45-856a-66fa7104e19d","Type":"ContainerDied","Data":"5a5594183935f0056eab7bf5280eafad42d59e502e9770b20f103eaeca683465"}
Nov 02 12:45:02.388902 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:02.388602    4942 scope.go:122] "RemoveContainer" containerID="5a5594183935f0056eab7bf5280eafad42d59e502e9770b20f103eaeca683465"
Nov 02 12:45:03.395646 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:03.395615    4942 generic.go:356] "Generic (PLEG): container finished" podID="9a27f0b8-53b2-4b45-856a-66fa7104e19d" containerID="aea466be0a72578dbfddc848cfc94468c9f68d55428c9c277414aa152dddaccf" exitCode=60
Nov 02 12:45:03.396056 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:03.395697    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp" event={"ID":"9a27f0b8-53b2-4b45-856a-66fa7104e19d","Type":"ContainerDied","Data":"aea466be0a72578dbfddc848cfc94468c9f68d55428c9c277414aa152dddaccf"}
Nov 02 12:45:03.396056 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:03.395760    4942 scope.go:122] "RemoveContainer" containerID="5a5594183935f0056eab7bf5280eafad42d59e502e9770b20f103eaeca683465"
Nov 02 12:45:03.396056 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:03.395970    4942 scope.go:122] "RemoveContainer" containerID="aea466be0a72578dbfddc848cfc94468c9f68d55428c9c277414aa152dddaccf"
Nov 02 12:45:03.396241 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:45:03.396217    4942 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp_odh-ai-gateway-infra(9a27f0b8-53b2-4b45-856a-66fa7104e19d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp" podUID="9a27f0b8-53b2-4b45-856a-66fa7104e19d"
Nov 02 12:45:04.403270 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:04.403236    4942 scope.go:122] "RemoveContainer" containerID="aea466be0a72578dbfddc848cfc94468c9f68d55428c9c277414aa152dddaccf"
Nov 02 12:45:04.403769 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:45:04.403523    4942 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp_odh-ai-gateway-infra(9a27f0b8-53b2-4b45-856a-66fa7104e19d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp" podUID="9a27f0b8-53b2-4b45-856a-66fa7104e19d"
Nov 02 12:45:17.535750 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:17.535715    4942 scope.go:122] "RemoveContainer" containerID="aea466be0a72578dbfddc848cfc94468c9f68d55428c9c277414aa152dddaccf"
Nov 02 12:45:18.478155 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:18.478125    4942 generic.go:356] "Generic (PLEG): container finished" podID="9a27f0b8-53b2-4b45-856a-66fa7104e19d" containerID="ce45981013d3dc22beceec44a1b6a2085e63717941769acd6071dfbb7baf8f76" exitCode=60
Nov 02 12:45:18.478324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:18.478203    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp" event={"ID":"9a27f0b8-53b2-4b45-856a-66fa7104e19d","Type":"ContainerDied","Data":"ce45981013d3dc22beceec44a1b6a2085e63717941769acd6071dfbb7baf8f76"}
Nov 02 12:45:18.478324 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:18.478279    4942 scope.go:122] "RemoveContainer" containerID="aea466be0a72578dbfddc848cfc94468c9f68d55428c9c277414aa152dddaccf"
Nov 02 12:45:18.478626 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:18.478603    4942 scope.go:122] "RemoveContainer" containerID="ce45981013d3dc22beceec44a1b6a2085e63717941769acd6071dfbb7baf8f76"
Nov 02 12:45:18.478850 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:45:18.478823    4942 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp_odh-ai-gateway-infra(9a27f0b8-53b2-4b45-856a-66fa7104e19d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp" podUID="9a27f0b8-53b2-4b45-856a-66fa7104e19d"
Nov 02 12:45:19.604174 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:19.604136    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp"]
Nov 02 12:45:19.748405 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:19.748383    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp"
Nov 02 12:45:19.864545 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:19.864457    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9a27f0b8-53b2-4b45-856a-66fa7104e19d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a27f0b8-53b2-4b45-856a-66fa7104e19d-maas-api-service-ca\") pod \"9a27f0b8-53b2-4b45-856a-66fa7104e19d\" (UID: \"9a27f0b8-53b2-4b45-856a-66fa7104e19d\") "
Nov 02 12:45:19.864708 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:19.864545    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a27f0b8-53b2-4b45-856a-66fa7104e19d-kube-api-access-b47x8\" (UniqueName: \"kubernetes.io/projected/9a27f0b8-53b2-4b45-856a-66fa7104e19d-kube-api-access-b47x8\") pod \"9a27f0b8-53b2-4b45-856a-66fa7104e19d\" (UID: \"9a27f0b8-53b2-4b45-856a-66fa7104e19d\") "
Nov 02 12:45:19.865746 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:19.865716    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a27f0b8-53b2-4b45-856a-66fa7104e19d-maas-api-service-ca" pod "9a27f0b8-53b2-4b45-856a-66fa7104e19d" (UID: "9a27f0b8-53b2-4b45-856a-66fa7104e19d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 12:45:19.876959 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:19.867440    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a27f0b8-53b2-4b45-856a-66fa7104e19d-kube-api-access-b47x8" pod "9a27f0b8-53b2-4b45-856a-66fa7104e19d" (UID: "9a27f0b8-53b2-4b45-856a-66fa7104e19d"). InnerVolumeSpecName "kube-api-access-b47x8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 12:45:19.966209 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:19.966178    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b47x8\" (UniqueName: \"kubernetes.io/projected/9a27f0b8-53b2-4b45-856a-66fa7104e19d-kube-api-access-b47x8\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:45:19.966209 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:19.966207    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a27f0b8-53b2-4b45-856a-66fa7104e19d-maas-api-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 12:45:20.491818 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:20.491780    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp" event={"ID":"9a27f0b8-53b2-4b45-856a-66fa7104e19d","Type":"ContainerDied","Data":"5c7c7bcc80f054733aed73b109a9adb8db52306a50b246bdda1609015120281c"}
Nov 02 12:45:20.491989 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:20.491835    4942 scope.go:122] "RemoveContainer" containerID="ce45981013d3dc22beceec44a1b6a2085e63717941769acd6071dfbb7baf8f76"
Nov 02 12:45:20.491989 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:20.491790    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp"
Nov 02 12:45:20.655558 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:20.655530    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp"]
Nov 02 12:45:20.659589 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:20.659565    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-a-6ac03f-29893725-zwzsp"]
Nov 02 12:45:21.541539 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:45:21.541496    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a27f0b8-53b2-4b45-856a-66fa7104e19d" path="/var/lib/kubelet/pods/9a27f0b8-53b2-4b45-856a-66fa7104e19d/volumes"
Nov 02 12:45:30.982953 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:45:30.982919    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:46:00.982074 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:46:00.982038    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:46:13.984928 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:46:13.984894    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:46:13.995545 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:46:13.995522    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:46:30.986645 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:46:30.986550    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:47:00.986112 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:47:00.986067    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:47:30.983200 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:47:30.983154    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:48:00.985973 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:48:00.985898    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:48:30.982899 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:48:30.982856    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:49:00.981949 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:49:00.981905    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:49:30.982913 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:49:30.982831    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:50:00.982377 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:50:00.982340    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:50:30.982136 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:50:30.982100    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:51:00.982033 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:51:00.981948    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:51:14.020526 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:51:14.020496    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:51:14.033842 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:51:14.033813    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:51:30.982738 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:51:30.982697    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:52:00.983010 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:52:00.982965    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:52:30.983399 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:52:30.983312    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:53:00.982202 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:53:00.982166    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:53:30.983335 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:53:30.983284    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:54:00.983224 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:54:00.983149    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:54:30.983209 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:54:30.983172    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:55:00.982862 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:55:00.982826    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:55:30.982021 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:55:30.981986    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:56:00.981808 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:56:00.981775    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:56:14.059246 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:56:14.059211    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:56:14.070569 ip-10-0-135-24 kubenswrapper[4942]: I1102 12:56:14.070546    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 12:56:30.981353 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:56:30.981318    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:57:00.981890 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:57:00.981815    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:57:30.984606 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:57:30.984570    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:58:00.981902 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:58:00.981866    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:58:30.983569 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:58:30.983488    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:59:00.983206 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:59:00.983167    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 12:59:30.982670 ip-10-0-135-24 kubenswrapper[4942]: E1102 12:59:30.982634    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:00:00.171235 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.171145    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt"]
Nov 02 13:00:00.171815 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.171792    4942 cpu_manager.go:414] "containerMap: removing container" podUID="9a27f0b8-53b2-4b45-856a-66fa7104e19d" containerName="cleanup"
Nov 02 13:00:00.171899 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.171819    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a27f0b8-53b2-4b45-856a-66fa7104e19d" containerName="cleanup"
Nov 02 13:00:00.171899 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.171864    4942 cpu_manager.go:414] "containerMap: removing container" podUID="29c6e230-f0a0-4d1f-bb69-db84edee3f9e" containerName="istio-proxy"
Nov 02 13:00:00.171899 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.171872    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29c6e230-f0a0-4d1f-bb69-db84edee3f9e" containerName="istio-proxy"
Nov 02 13:00:00.171899 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.171887    4942 cpu_manager.go:414] "containerMap: removing container" podUID="9a27f0b8-53b2-4b45-856a-66fa7104e19d" containerName="cleanup"
Nov 02 13:00:00.171899 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.171895    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a27f0b8-53b2-4b45-856a-66fa7104e19d" containerName="cleanup"
Nov 02 13:00:00.172155 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.171905    4942 cpu_manager.go:414] "containerMap: removing container" podUID="9a27f0b8-53b2-4b45-856a-66fa7104e19d" containerName="cleanup"
Nov 02 13:00:00.172155 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.171915    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a27f0b8-53b2-4b45-856a-66fa7104e19d" containerName="cleanup"
Nov 02 13:00:00.172155 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.171927    4942 cpu_manager.go:414] "containerMap: removing container" podUID="c221cddd-0f93-4f81-9b40-1d017ad063fe" containerName="istio-proxy"
Nov 02 13:00:00.172155 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.171936    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c221cddd-0f93-4f81-9b40-1d017ad063fe" containerName="istio-proxy"
Nov 02 13:00:00.172155 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.172048    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a27f0b8-53b2-4b45-856a-66fa7104e19d" containerName="cleanup"
Nov 02 13:00:00.172155 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.172065    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a27f0b8-53b2-4b45-856a-66fa7104e19d" containerName="cleanup"
Nov 02 13:00:00.172155 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.172074    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a27f0b8-53b2-4b45-856a-66fa7104e19d" containerName="cleanup"
Nov 02 13:00:00.172155 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.172082    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="c221cddd-0f93-4f81-9b40-1d017ad063fe" containerName="istio-proxy"
Nov 02 13:00:00.172155 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.172093    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="29c6e230-f0a0-4d1f-bb69-db84edee3f9e" containerName="istio-proxy"
Nov 02 13:00:00.175267 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.175245    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt"
Nov 02 13:00:00.195874 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.195846    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmb8x\" (UniqueName: \"kubernetes.io/projected/1624b550-3505-45e1-8107-3d84c8cf267b-kube-api-access-hmb8x\") pod \"maas-api-key-cleanup-29893740-dpmvt\" (UID: \"1624b550-3505-45e1-8107-3d84c8cf267b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt"
Nov 02 13:00:00.195979 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.195886    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1624b550-3505-45e1-8107-3d84c8cf267b-maas-api-service-ca\") pod \"maas-api-key-cleanup-29893740-dpmvt\" (UID: \"1624b550-3505-45e1-8107-3d84c8cf267b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt"
Nov 02 13:00:00.229228 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.229201    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nbw2z\""
Nov 02 13:00:00.280968 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.280933    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt"]
Nov 02 13:00:00.297218 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.297186    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmb8x\" (UniqueName: \"kubernetes.io/projected/1624b550-3505-45e1-8107-3d84c8cf267b-kube-api-access-hmb8x\") pod \"maas-api-key-cleanup-29893740-dpmvt\" (UID: \"1624b550-3505-45e1-8107-3d84c8cf267b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt"
Nov 02 13:00:00.297384 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.297235    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1624b550-3505-45e1-8107-3d84c8cf267b-maas-api-service-ca\") pod \"maas-api-key-cleanup-29893740-dpmvt\" (UID: \"1624b550-3505-45e1-8107-3d84c8cf267b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt"
Nov 02 13:00:00.297921 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.297901    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1624b550-3505-45e1-8107-3d84c8cf267b-maas-api-service-ca\") pod \"maas-api-key-cleanup-29893740-dpmvt\" (UID: \"1624b550-3505-45e1-8107-3d84c8cf267b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt"
Nov 02 13:00:00.357767 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.357735    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmb8x\" (UniqueName: \"kubernetes.io/projected/1624b550-3505-45e1-8107-3d84c8cf267b-kube-api-access-hmb8x\") pod \"maas-api-key-cleanup-29893740-dpmvt\" (UID: \"1624b550-3505-45e1-8107-3d84c8cf267b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt"
Nov 02 13:00:00.487585 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.487519    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt"
Nov 02 13:00:00.861072 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:00.861042    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt"]
Nov 02 13:00:00.861674 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:00:00.861646    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1624b550_3505_45e1_8107_3d84c8cf267b.slice/crio-91d9d89cf612d51df2e5e23d96cf1ee1f89258ecf8bc7c78e006b4dda6a39d3b WatchSource:0}: Error finding container 91d9d89cf612d51df2e5e23d96cf1ee1f89258ecf8bc7c78e006b4dda6a39d3b: Status 404 returned error can't find the container with id 91d9d89cf612d51df2e5e23d96cf1ee1f89258ecf8bc7c78e006b4dda6a39d3b
Nov 02 13:00:00.982322 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:00:00.982270    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:00:01.145440 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:01.145361    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt" event={"ID":"1624b550-3505-45e1-8107-3d84c8cf267b","Type":"ContainerStarted","Data":"91d9d89cf612d51df2e5e23d96cf1ee1f89258ecf8bc7c78e006b4dda6a39d3b"}
Nov 02 13:00:02.152652 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:02.152609    4942 generic.go:356] "Generic (PLEG): container finished" podID="1624b550-3505-45e1-8107-3d84c8cf267b" containerID="92f84b11dd20c673725567796fae25eff4686f5f767744f6058bd2f246cce5d5" exitCode=60
Nov 02 13:00:02.153113 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:02.152670    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt" event={"ID":"1624b550-3505-45e1-8107-3d84c8cf267b","Type":"ContainerDied","Data":"92f84b11dd20c673725567796fae25eff4686f5f767744f6058bd2f246cce5d5"}
Nov 02 13:00:02.153113 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:02.152874    4942 scope.go:122] "RemoveContainer" containerID="92f84b11dd20c673725567796fae25eff4686f5f767744f6058bd2f246cce5d5"
Nov 02 13:00:03.159689 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:03.159597    4942 generic.go:356] "Generic (PLEG): container finished" podID="1624b550-3505-45e1-8107-3d84c8cf267b" containerID="5f9732f5975da82b9cd45ce0449702fec3093629f7336fbf237318efeb965cc8" exitCode=60
Nov 02 13:00:03.159689 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:03.159667    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt" event={"ID":"1624b550-3505-45e1-8107-3d84c8cf267b","Type":"ContainerDied","Data":"5f9732f5975da82b9cd45ce0449702fec3093629f7336fbf237318efeb965cc8"}
Nov 02 13:00:03.160282 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:03.159760    4942 scope.go:122] "RemoveContainer" containerID="92f84b11dd20c673725567796fae25eff4686f5f767744f6058bd2f246cce5d5"
Nov 02 13:00:03.160282 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:03.159995    4942 scope.go:122] "RemoveContainer" containerID="5f9732f5975da82b9cd45ce0449702fec3093629f7336fbf237318efeb965cc8"
Nov 02 13:00:03.160622 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:00:03.160331    4942 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29893740-dpmvt_odh-ai-gateway-infra(1624b550-3505-45e1-8107-3d84c8cf267b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt" podUID="1624b550-3505-45e1-8107-3d84c8cf267b"
Nov 02 13:00:04.166535 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:04.166507    4942 scope.go:122] "RemoveContainer" containerID="5f9732f5975da82b9cd45ce0449702fec3093629f7336fbf237318efeb965cc8"
Nov 02 13:00:04.166943 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:00:04.166724    4942 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29893740-dpmvt_odh-ai-gateway-infra(1624b550-3505-45e1-8107-3d84c8cf267b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt" podUID="1624b550-3505-45e1-8107-3d84c8cf267b"
Nov 02 13:00:15.536563 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:15.536532    4942 scope.go:122] "RemoveContainer" containerID="5f9732f5975da82b9cd45ce0449702fec3093629f7336fbf237318efeb965cc8"
Nov 02 13:00:16.253974 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:16.253875    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt" event={"ID":"1624b550-3505-45e1-8107-3d84c8cf267b","Type":"ContainerStarted","Data":"d7209a876ae26232a34de12cf67ba27d5f6b5824c319f13a22acf5f54347d721"}
Nov 02 13:00:16.370220 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:16.370172    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt" podStartSLOduration=16.370158833 podStartE2EDuration="16.370158833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:00:16.368963298 +0000 UTC m=+3842.940313056" watchObservedRunningTime="2026-11-02 13:00:16.370158833 +0000 UTC m=+3842.941508589"
Nov 02 13:00:17.261538 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:17.261507    4942 generic.go:356] "Generic (PLEG): container finished" podID="1624b550-3505-45e1-8107-3d84c8cf267b" containerID="d7209a876ae26232a34de12cf67ba27d5f6b5824c319f13a22acf5f54347d721" exitCode=60
Nov 02 13:00:17.262005 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:17.261550    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt" event={"ID":"1624b550-3505-45e1-8107-3d84c8cf267b","Type":"ContainerDied","Data":"d7209a876ae26232a34de12cf67ba27d5f6b5824c319f13a22acf5f54347d721"}
Nov 02 13:00:17.262005 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:17.261597    4942 scope.go:122] "RemoveContainer" containerID="5f9732f5975da82b9cd45ce0449702fec3093629f7336fbf237318efeb965cc8"
Nov 02 13:00:17.262005 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:17.261895    4942 scope.go:122] "RemoveContainer" containerID="d7209a876ae26232a34de12cf67ba27d5f6b5824c319f13a22acf5f54347d721"
Nov 02 13:00:17.262172 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:00:17.262121    4942 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29893740-dpmvt_odh-ai-gateway-infra(1624b550-3505-45e1-8107-3d84c8cf267b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt" podUID="1624b550-3505-45e1-8107-3d84c8cf267b"
Nov 02 13:00:17.390399 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:17.390362    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt"]
Nov 02 13:00:18.412566 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:18.412539    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt"
Nov 02 13:00:18.572853 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:18.572762    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1624b550-3505-45e1-8107-3d84c8cf267b-kube-api-access-hmb8x\" (UniqueName: \"kubernetes.io/projected/1624b550-3505-45e1-8107-3d84c8cf267b-kube-api-access-hmb8x\") pod \"1624b550-3505-45e1-8107-3d84c8cf267b\" (UID: \"1624b550-3505-45e1-8107-3d84c8cf267b\") "
Nov 02 13:00:18.573033 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:18.572968    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1624b550-3505-45e1-8107-3d84c8cf267b-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1624b550-3505-45e1-8107-3d84c8cf267b-maas-api-service-ca\") pod \"1624b550-3505-45e1-8107-3d84c8cf267b\" (UID: \"1624b550-3505-45e1-8107-3d84c8cf267b\") "
Nov 02 13:00:18.573317 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:18.573277    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1624b550-3505-45e1-8107-3d84c8cf267b-maas-api-service-ca" pod "1624b550-3505-45e1-8107-3d84c8cf267b" (UID: "1624b550-3505-45e1-8107-3d84c8cf267b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:00:18.575449 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:18.575416    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1624b550-3505-45e1-8107-3d84c8cf267b-kube-api-access-hmb8x" pod "1624b550-3505-45e1-8107-3d84c8cf267b" (UID: "1624b550-3505-45e1-8107-3d84c8cf267b"). InnerVolumeSpecName "kube-api-access-hmb8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:00:18.674186 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:18.674152    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1624b550-3505-45e1-8107-3d84c8cf267b-maas-api-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:00:18.674186 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:18.674182    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmb8x\" (UniqueName: \"kubernetes.io/projected/1624b550-3505-45e1-8107-3d84c8cf267b-kube-api-access-hmb8x\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:00:19.177051 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.177016    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"]
Nov 02 13:00:19.177437 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.177424    4942 cpu_manager.go:414] "containerMap: removing container" podUID="1624b550-3505-45e1-8107-3d84c8cf267b" containerName="cleanup"
Nov 02 13:00:19.177488 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.177438    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1624b550-3505-45e1-8107-3d84c8cf267b" containerName="cleanup"
Nov 02 13:00:19.177488 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.177452    4942 cpu_manager.go:414] "containerMap: removing container" podUID="1624b550-3505-45e1-8107-3d84c8cf267b" containerName="cleanup"
Nov 02 13:00:19.177488 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.177457    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1624b550-3505-45e1-8107-3d84c8cf267b" containerName="cleanup"
Nov 02 13:00:19.177488 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.177465    4942 cpu_manager.go:414] "containerMap: removing container" podUID="1624b550-3505-45e1-8107-3d84c8cf267b" containerName="cleanup"
Nov 02 13:00:19.177488 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.177470    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1624b550-3505-45e1-8107-3d84c8cf267b" containerName="cleanup"
Nov 02 13:00:19.177643 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.177558    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="1624b550-3505-45e1-8107-3d84c8cf267b" containerName="cleanup"
Nov 02 13:00:19.177643 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.177565    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="1624b550-3505-45e1-8107-3d84c8cf267b" containerName="cleanup"
Nov 02 13:00:19.177643 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.177573    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="1624b550-3505-45e1-8107-3d84c8cf267b" containerName="cleanup"
Nov 02 13:00:19.180662 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.180645    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.275674 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.275644    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt"
Nov 02 13:00:19.275841 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.275687    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt" event={"ID":"1624b550-3505-45e1-8107-3d84c8cf267b","Type":"ContainerDied","Data":"91d9d89cf612d51df2e5e23d96cf1ee1f89258ecf8bc7c78e006b4dda6a39d3b"}
Nov 02 13:00:19.275841 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.275747    4942 scope.go:122] "RemoveContainer" containerID="d7209a876ae26232a34de12cf67ba27d5f6b5824c319f13a22acf5f54347d721"
Nov 02 13:00:19.279624 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.279599    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.279745 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.279690    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eab2ec94-23e3-4528-8f27-2e9f16498268-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.279745 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.279738    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.279886 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.279755    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.279886 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.279788    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-457qt\" (UniqueName: \"kubernetes.io/projected/eab2ec94-23e3-4528-8f27-2e9f16498268-kube-api-access-457qt\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.279886 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.279864    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.280031 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.279900    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.280031 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.279929    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.280031 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.279974    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.285459 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.285442    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"]
Nov 02 13:00:19.380632 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.380605    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.380766 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.380639    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.380766 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.380667    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.380893 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.380819    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.380954 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.380909    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eab2ec94-23e3-4528-8f27-2e9f16498268-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.381005 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.380960    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.381005 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.380975    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.381104 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.381022    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.381104 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.381079    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.381218 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.381127    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-457qt\" (UniqueName: \"kubernetes.io/projected/eab2ec94-23e3-4528-8f27-2e9f16498268-kube-api-access-457qt\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.381218 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.381154    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.381218 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.381189    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.381559 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.381538    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.381638 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.381622    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eab2ec94-23e3-4528-8f27-2e9f16498268-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.383108 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.383088    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.383209 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.383120    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.416141 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.416121    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt"]
Nov 02 13:00:19.423441 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.423421    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29893740-dpmvt"]
Nov 02 13:00:19.448631 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.448580    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.448765 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.448747    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-457qt\" (UniqueName: \"kubernetes.io/projected/eab2ec94-23e3-4528-8f27-2e9f16498268-kube-api-access-457qt\") pod \"maas-default-gateway-openshift-default-6dcf757899-xhwwv\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.492942 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.492916    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:19.544044 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.544007    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1624b550-3505-45e1-8107-3d84c8cf267b" path="/var/lib/kubelet/pods/1624b550-3505-45e1-8107-3d84c8cf267b/volumes"
Nov 02 13:00:19.918728 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.918695    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"]
Nov 02 13:00:19.920990 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:00:19.920963    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeab2ec94_23e3_4528_8f27_2e9f16498268.slice/crio-63cb195fcc5d38317fe21bfab86d02ed6664c7b54a3987fc84249baa41420170 WatchSource:0}: Error finding container 63cb195fcc5d38317fe21bfab86d02ed6664c7b54a3987fc84249baa41420170: Status 404 returned error can't find the container with id 63cb195fcc5d38317fe21bfab86d02ed6664c7b54a3987fc84249baa41420170
Nov 02 13:00:19.923189 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.923158    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:00:19.923255 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.923223    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:00:19.923255 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:19.923251    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:00:20.283146 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:20.283058    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv" event={"ID":"eab2ec94-23e3-4528-8f27-2e9f16498268","Type":"ContainerStarted","Data":"43d8cde9fdf73327439b0c8e979145e4c9243015fb8561c177aeb7555209ff5d"}
Nov 02 13:00:20.283146 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:20.283116    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv" event={"ID":"eab2ec94-23e3-4528-8f27-2e9f16498268","Type":"ContainerStarted","Data":"63cb195fcc5d38317fe21bfab86d02ed6664c7b54a3987fc84249baa41420170"}
Nov 02 13:00:20.412320 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:20.410157    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv" podStartSLOduration=1.410136361 podStartE2EDuration="1.410136361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:00:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:00:20.407877137 +0000 UTC m=+3846.979226895" watchObservedRunningTime="2026-11-02 13:00:20.410136361 +0000 UTC m=+3846.981486119"
Nov 02 13:00:20.493968 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:20.493935    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:20.495123 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:20.495100    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-xhwwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 02 13:00:20.495194 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:20.495154    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv" podUID="eab2ec94-23e3-4528-8f27-2e9f16498268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 02 13:00:21.493959 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:21.493927    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-xhwwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 02 13:00:21.494145 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:21.493992    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv" podUID="eab2ec94-23e3-4528-8f27-2e9f16498268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 02 13:00:22.493977 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:22.493941    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-xhwwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 02 13:00:22.494173 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:22.494001    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv" podUID="eab2ec94-23e3-4528-8f27-2e9f16498268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 02 13:00:23.494082 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:23.494035    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-xhwwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 02 13:00:23.494327 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:23.494137    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv" podUID="eab2ec94-23e3-4528-8f27-2e9f16498268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 02 13:00:24.493990 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:24.493951    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-xhwwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 02 13:00:24.494136 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:24.494020    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv" podUID="eab2ec94-23e3-4528-8f27-2e9f16498268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 02 13:00:25.493782 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:25.493738    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-xhwwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 02 13:00:25.494322 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:25.493821    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv" podUID="eab2ec94-23e3-4528-8f27-2e9f16498268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 02 13:00:26.493445 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:26.493408    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-xhwwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 02 13:00:26.493632 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:26.493458    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv" podUID="eab2ec94-23e3-4528-8f27-2e9f16498268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 02 13:00:27.493516 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:27.493479    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-xhwwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 02 13:00:27.494017 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:27.493569    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv" podUID="eab2ec94-23e3-4528-8f27-2e9f16498268" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 02 13:00:29.225112 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:29.225082    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:29.225606 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:29.225500    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:29.226360 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:00:29.226337    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:00:30.982490 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:00:30.982456    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:01:00.981736 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:01:00.981698    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:01:04.541219 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.541186    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"]
Nov 02 13:01:04.544981 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.544962    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.635490 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.635455    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.635650 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.635554    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.635650 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.635593    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.635650 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.635612    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.635650 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.635630    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.635886 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.635791    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.635886 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.635858    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lwq5\" (UniqueName: \"kubernetes.io/projected/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-kube-api-access-8lwq5\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.635995 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.635955    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.636050 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.636015    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.736750 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.736712    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.736939 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.736757    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lwq5\" (UniqueName: \"kubernetes.io/projected/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-kube-api-access-8lwq5\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.736939 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.736811    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.736939 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.736843    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.736939 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.736887    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.736939 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.736927    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.737209 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.736956    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.737209 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.736983    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.737350 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.737286    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.737431 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.737405    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.737491 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.737468    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.737552 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.737519    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.737657 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.737635    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.737988 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.737965    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.739197 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.739172    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.739393 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.739372    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.751084 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.751060    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"]
Nov 02 13:01:04.798266 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.798196    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.799052 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.799030    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lwq5\" (UniqueName: \"kubernetes.io/projected/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-kube-api-access-8lwq5\") pod \"maas-default-gateway-openshift-default-6dcf757899-nchmc\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:04.858161 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:04.858139    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:05.285447 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:05.285422    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"]
Nov 02 13:01:05.287449 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:01:05.287417    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6a9f536_1785_4c38_9fc1_6a2cb5dff854.slice/crio-ff5dd8c0af6078bb437e90289feaaba076ccad2b0f491aefb7b8dc6867264148 WatchSource:0}: Error finding container ff5dd8c0af6078bb437e90289feaaba076ccad2b0f491aefb7b8dc6867264148: Status 404 returned error can't find the container with id ff5dd8c0af6078bb437e90289feaaba076ccad2b0f491aefb7b8dc6867264148
Nov 02 13:01:05.289633 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:05.289599    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:01:05.289716 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:05.289670    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:01:05.289716 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:05.289701    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:01:05.577124 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:05.577027    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc" event={"ID":"a6a9f536-1785-4c38-9fc1-6a2cb5dff854","Type":"ContainerStarted","Data":"3da51a5a058c729105abfde91436482ebc5e024d572ad712f69450c7ef222f54"}
Nov 02 13:01:05.577124 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:05.577083    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc" event={"ID":"a6a9f536-1785-4c38-9fc1-6a2cb5dff854","Type":"ContainerStarted","Data":"ff5dd8c0af6078bb437e90289feaaba076ccad2b0f491aefb7b8dc6867264148"}
Nov 02 13:01:05.699872 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:05.699801    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc" podStartSLOduration=1.699782565 podStartE2EDuration="1.699782565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:01:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:01:05.697794295 +0000 UTC m=+3892.269144053" watchObservedRunningTime="2026-11-02 13:01:05.699782565 +0000 UTC m=+3892.271132328"
Nov 02 13:01:05.866343 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:05.858981    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:06.877556 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:06.877506    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-nchmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 02 13:01:06.878150 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:06.877620    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc" podUID="a6a9f536-1785-4c38-9fc1-6a2cb5dff854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 02 13:01:07.858673 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:07.858637    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-nchmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 02 13:01:07.858877 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:07.858702    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc" podUID="a6a9f536-1785-4c38-9fc1-6a2cb5dff854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 02 13:01:08.859187 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:08.859149    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-nchmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 02 13:01:08.859817 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:08.859786    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc" podUID="a6a9f536-1785-4c38-9fc1-6a2cb5dff854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 02 13:01:09.858659 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:09.858621    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-nchmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 02 13:01:09.859007 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:09.858977    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc" podUID="a6a9f536-1785-4c38-9fc1-6a2cb5dff854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 02 13:01:10.873099 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:10.872655    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-nchmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 02 13:01:10.873099 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:10.872717    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc" podUID="a6a9f536-1785-4c38-9fc1-6a2cb5dff854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 02 13:01:11.859140 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:11.859106    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-nchmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 02 13:01:11.859336 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:11.859159    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc" podUID="a6a9f536-1785-4c38-9fc1-6a2cb5dff854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 02 13:01:12.858899 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:12.858863    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-nchmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 02 13:01:12.859321 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:12.858948    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc" podUID="a6a9f536-1785-4c38-9fc1-6a2cb5dff854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 02 13:01:14.100803 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:14.100771    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:01:14.112389 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:14.112368    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:01:14.593866 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:14.593828    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:14.594277 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:14.594251    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:14.595201 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:01:14.595182    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:01:30.983633 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:01:30.983558    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:02:00.982790 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:02:00.982751    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:02:30.984865 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:02:30.984817    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:03:00.982664 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:03:00.982586    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:03:30.983542 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:03:30.983497    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:04:00.989201 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:04:00.989162    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:04:30.983538 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:04:30.983456    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:05:00.983460 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:05:00.983423    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:05:30.982165 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:05:30.982130    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:06:00.983416 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:06:00.983334    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:06:14.139704 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:14.139671    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:06:14.152846 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:14.152823    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:06:21.926730 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:21.926693    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"]
Nov 02 13:06:21.927256 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:21.926899    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc" podUID="a6a9f536-1785-4c38-9fc1-6a2cb5dff854" containerName="istio-proxy" containerID="cri-o://3da51a5a058c729105abfde91436482ebc5e024d572ad712f69450c7ef222f54" gracePeriod=30
Nov 02 13:06:27.176385 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.176361    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:06:27.228526 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.228449    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-workload-socket\") pod \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") "
Nov 02 13:06:27.228655 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.228536    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-data\") pod \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") "
Nov 02 13:06:27.228655 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.228560    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-workload-certs\") pod \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") "
Nov 02 13:06:27.228655 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.228587    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-envoy\") pod \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") "
Nov 02 13:06:27.228655 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.228642    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-kube-api-access-8lwq5\" (UniqueName: \"kubernetes.io/projected/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-kube-api-access-8lwq5\") pod \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") "
Nov 02 13:06:27.228857 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.228682    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-credential-socket\") pod \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") "
Nov 02 13:06:27.228857 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.228748    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-workload-socket" pod "a6a9f536-1785-4c38-9fc1-6a2cb5dff854" (UID: "a6a9f536-1785-4c38-9fc1-6a2cb5dff854"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:06:27.228857 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.228779    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-podinfo\") pod \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") "
Nov 02 13:06:27.228857 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.228782    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-data" pod "a6a9f536-1785-4c38-9fc1-6a2cb5dff854" (UID: "a6a9f536-1785-4c38-9fc1-6a2cb5dff854"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:06:27.229069 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.228900    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-workload-certs" pod "a6a9f536-1785-4c38-9fc1-6a2cb5dff854" (UID: "a6a9f536-1785-4c38-9fc1-6a2cb5dff854"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:06:27.229069 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.229032    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-credential-socket" pod "a6a9f536-1785-4c38-9fc1-6a2cb5dff854" (UID: "a6a9f536-1785-4c38-9fc1-6a2cb5dff854"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:06:27.229069 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.229042    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-token\" (UniqueName: \"kubernetes.io/projected/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-token\") pod \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") "
Nov 02 13:06:27.229257 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.229122    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istiod-ca-cert\") pod \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\" (UID: \"a6a9f536-1785-4c38-9fc1-6a2cb5dff854\") "
Nov 02 13:06:27.229503 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.229482    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:06:27.229634 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.229557    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:06:27.229634 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.229578    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:06:27.229634 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.229592    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:06:27.229634 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.229499    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istiod-ca-cert" pod "a6a9f536-1785-4c38-9fc1-6a2cb5dff854" (UID: "a6a9f536-1785-4c38-9fc1-6a2cb5dff854"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:06:27.231425 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.231402    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-envoy" pod "a6a9f536-1785-4c38-9fc1-6a2cb5dff854" (UID: "a6a9f536-1785-4c38-9fc1-6a2cb5dff854"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:06:27.231425 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.231415    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-kube-api-access-8lwq5" pod "a6a9f536-1785-4c38-9fc1-6a2cb5dff854" (UID: "a6a9f536-1785-4c38-9fc1-6a2cb5dff854"). InnerVolumeSpecName "kube-api-access-8lwq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:06:27.231555 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.231463    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-podinfo" pod "a6a9f536-1785-4c38-9fc1-6a2cb5dff854" (UID: "a6a9f536-1785-4c38-9fc1-6a2cb5dff854"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 13:06:27.231657 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.231639    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-token" pod "a6a9f536-1785-4c38-9fc1-6a2cb5dff854" (UID: "a6a9f536-1785-4c38-9fc1-6a2cb5dff854"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:06:27.330268 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.330238    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:06:27.330268 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.330267    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lwq5\" (UniqueName: \"kubernetes.io/projected/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-kube-api-access-8lwq5\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:06:27.330524 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.330277    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:06:27.330524 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.330286    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:06:27.330524 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.330322    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6a9f536-1785-4c38-9fc1-6a2cb5dff854-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:06:27.405889 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.405854    4942 generic.go:356] "Generic (PLEG): container finished" podID="a6a9f536-1785-4c38-9fc1-6a2cb5dff854" containerID="3da51a5a058c729105abfde91436482ebc5e024d572ad712f69450c7ef222f54" exitCode=0
Nov 02 13:06:27.406048 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.405905    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"
Nov 02 13:06:27.406048 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.405936    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc" event={"ID":"a6a9f536-1785-4c38-9fc1-6a2cb5dff854","Type":"ContainerDied","Data":"3da51a5a058c729105abfde91436482ebc5e024d572ad712f69450c7ef222f54"}
Nov 02 13:06:27.406048 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.406002    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc" event={"ID":"a6a9f536-1785-4c38-9fc1-6a2cb5dff854","Type":"ContainerDied","Data":"ff5dd8c0af6078bb437e90289feaaba076ccad2b0f491aefb7b8dc6867264148"}
Nov 02 13:06:27.406048 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.406024    4942 scope.go:122] "RemoveContainer" containerID="3da51a5a058c729105abfde91436482ebc5e024d572ad712f69450c7ef222f54"
Nov 02 13:06:27.416573 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.416553    4942 scope.go:122] "RemoveContainer" containerID="3da51a5a058c729105abfde91436482ebc5e024d572ad712f69450c7ef222f54"
Nov 02 13:06:27.416862 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:06:27.416833    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3da51a5a058c729105abfde91436482ebc5e024d572ad712f69450c7ef222f54\": container with ID starting with 3da51a5a058c729105abfde91436482ebc5e024d572ad712f69450c7ef222f54 not found: ID does not exist" containerID="3da51a5a058c729105abfde91436482ebc5e024d572ad712f69450c7ef222f54"
Nov 02 13:06:27.416933 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.416875    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3da51a5a058c729105abfde91436482ebc5e024d572ad712f69450c7ef222f54"} err="failed to get container status \"3da51a5a058c729105abfde91436482ebc5e024d572ad712f69450c7ef222f54\": rpc error: code = NotFound desc = could not find container \"3da51a5a058c729105abfde91436482ebc5e024d572ad712f69450c7ef222f54\": container with ID starting with 3da51a5a058c729105abfde91436482ebc5e024d572ad712f69450c7ef222f54 not found: ID does not exist"
Nov 02 13:06:27.573074 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.573004    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"]
Nov 02 13:06:27.577128 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:27.577102    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-nchmc"]
Nov 02 13:06:29.542290 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:06:29.542259    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a6a9f536-1785-4c38-9fc1-6a2cb5dff854" path="/var/lib/kubelet/pods/a6a9f536-1785-4c38-9fc1-6a2cb5dff854/volumes"
Nov 02 13:06:30.984690 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:06:30.984648    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:07:00.983031 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:07:00.982992    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:07:30.988548 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:07:30.988461    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:07:42.742731 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:42.742692    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj"]
Nov 02 13:07:42.743169 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:42.743118    4942 cpu_manager.go:414] "containerMap: removing container" podUID="a6a9f536-1785-4c38-9fc1-6a2cb5dff854" containerName="istio-proxy"
Nov 02 13:07:42.743169 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:42.743132    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a6a9f536-1785-4c38-9fc1-6a2cb5dff854" containerName="istio-proxy"
Nov 02 13:07:42.743243 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:42.743235    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="a6a9f536-1785-4c38-9fc1-6a2cb5dff854" containerName="istio-proxy"
Nov 02 13:07:42.746193 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:42.746177    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj"
Nov 02 13:07:42.800560 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:42.800526    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-extmod-shr-user-1125bd\"/\"openshift-service-ca.crt\""
Nov 02 13:07:42.800748 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:42.800609    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-extmod-shr-user-1125bd\"/\"kube-root-ca.crt\""
Nov 02 13:07:42.849982 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:42.849955    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-extmod-shr-user-1125bd\"/\"e2e-usage-sim-eb01cf-cert\""
Nov 02 13:07:42.850125 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:42.850056    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-extmod-shr-user-1125bd\"/\"default-dockercfg-nd9pn\""
Nov 02 13:07:42.852402 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:42.852378    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj"]
Nov 02 13:07:42.938627 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:42.938599    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/3e476d3d-1090-4701-8a38-fd60c3f0d715-tls\") pod \"e2e-usage-sim-eb01cf-558676877d-8tvmj\" (UID: \"3e476d3d-1090-4701-8a38-fd60c3f0d715\") " pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj"
Nov 02 13:07:42.938793 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:42.938636    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p9zs\" (UniqueName: \"kubernetes.io/projected/3e476d3d-1090-4701-8a38-fd60c3f0d715-kube-api-access-9p9zs\") pod \"e2e-usage-sim-eb01cf-558676877d-8tvmj\" (UID: \"3e476d3d-1090-4701-8a38-fd60c3f0d715\") " pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj"
Nov 02 13:07:43.039743 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:43.039673    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls\" (UniqueName: \"kubernetes.io/secret/3e476d3d-1090-4701-8a38-fd60c3f0d715-tls\") pod \"e2e-usage-sim-eb01cf-558676877d-8tvmj\" (UID: \"3e476d3d-1090-4701-8a38-fd60c3f0d715\") " pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj"
Nov 02 13:07:43.039743 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:43.039728    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p9zs\" (UniqueName: \"kubernetes.io/projected/3e476d3d-1090-4701-8a38-fd60c3f0d715-kube-api-access-9p9zs\") pod \"e2e-usage-sim-eb01cf-558676877d-8tvmj\" (UID: \"3e476d3d-1090-4701-8a38-fd60c3f0d715\") " pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj"
Nov 02 13:07:43.042229 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:43.042202    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls\" (UniqueName: \"kubernetes.io/secret/3e476d3d-1090-4701-8a38-fd60c3f0d715-tls\") pod \"e2e-usage-sim-eb01cf-558676877d-8tvmj\" (UID: \"3e476d3d-1090-4701-8a38-fd60c3f0d715\") " pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj"
Nov 02 13:07:43.101835 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:43.101798    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p9zs\" (UniqueName: \"kubernetes.io/projected/3e476d3d-1090-4701-8a38-fd60c3f0d715-kube-api-access-9p9zs\") pod \"e2e-usage-sim-eb01cf-558676877d-8tvmj\" (UID: \"3e476d3d-1090-4701-8a38-fd60c3f0d715\") " pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj"
Nov 02 13:07:43.358625 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:43.358594    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj"
Nov 02 13:07:43.729179 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:43.729145    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj"]
Nov 02 13:07:43.731525 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:07:43.731490    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e476d3d_1090_4701_8a38_fd60c3f0d715.slice/crio-f87fa63158c8356bb92662489040f83f490b98be5b2870dbdc394e0763f76036 WatchSource:0}: Error finding container f87fa63158c8356bb92662489040f83f490b98be5b2870dbdc394e0763f76036: Status 404 returned error can't find the container with id f87fa63158c8356bb92662489040f83f490b98be5b2870dbdc394e0763f76036
Nov 02 13:07:43.817254 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:43.817222    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj" event={"ID":"3e476d3d-1090-4701-8a38-fd60c3f0d715","Type":"ContainerStarted","Data":"3c2d74a2600ef002c9cd89a2a55cfd93111a86327d3fd8ae7c3052d0c592ef39"}
Nov 02 13:07:43.817662 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:43.817279    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj"
Nov 02 13:07:43.817662 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:43.817291    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj" event={"ID":"3e476d3d-1090-4701-8a38-fd60c3f0d715","Type":"ContainerStarted","Data":"f87fa63158c8356bb92662489040f83f490b98be5b2870dbdc394e0763f76036"}
Nov 02 13:07:43.819177 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:43.819150    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj" podUID="3e476d3d-1090-4701-8a38-fd60c3f0d715" containerName="sim" probeResult="failure" output="Get \"https://10.132.0.117:8000/ready\": dial tcp 10.132.0.117:8000: connect: connection refused"
Nov 02 13:07:43.931255 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:43.931212    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj" podStartSLOduration=1.931198271 podStartE2EDuration="1.931198271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:07:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:07:43.930508645 +0000 UTC m=+4290.501858402" watchObservedRunningTime="2026-11-02 13:07:43.931198271 +0000 UTC m=+4290.502548029"
Nov 02 13:07:44.827767 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:07:44.827738    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj"
Nov 02 13:08:00.985349 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:08:00.985313    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:08:07.770202 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:07.770164    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"]
Nov 02 13:08:07.770761 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:07.770411    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv" podUID="eab2ec94-23e3-4528-8f27-2e9f16498268" containerName="istio-proxy" containerID="cri-o://43d8cde9fdf73327439b0c8e979145e4c9243015fb8561c177aeb7555209ff5d" gracePeriod=30
Nov 02 13:08:12.981450 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:12.981401    4942 generic.go:356] "Generic (PLEG): container finished" podID="eab2ec94-23e3-4528-8f27-2e9f16498268" containerID="43d8cde9fdf73327439b0c8e979145e4c9243015fb8561c177aeb7555209ff5d" exitCode=0
Nov 02 13:08:12.981450 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:12.981443    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv" event={"ID":"eab2ec94-23e3-4528-8f27-2e9f16498268","Type":"ContainerDied","Data":"43d8cde9fdf73327439b0c8e979145e4c9243015fb8561c177aeb7555209ff5d"}
Nov 02 13:08:13.016471 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.016446    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:08:13.113024 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.112995    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eab2ec94-23e3-4528-8f27-2e9f16498268-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eab2ec94-23e3-4528-8f27-2e9f16498268-istiod-ca-cert\") pod \"eab2ec94-23e3-4528-8f27-2e9f16498268\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") "
Nov 02 13:08:13.113199 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.113077    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-podinfo\") pod \"eab2ec94-23e3-4528-8f27-2e9f16498268\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") "
Nov 02 13:08:13.113199 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.113097    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-credential-socket\") pod \"eab2ec94-23e3-4528-8f27-2e9f16498268\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") "
Nov 02 13:08:13.113199 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.113177    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-data\") pod \"eab2ec94-23e3-4528-8f27-2e9f16498268\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") "
Nov 02 13:08:13.113406 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.113211    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-envoy\") pod \"eab2ec94-23e3-4528-8f27-2e9f16498268\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") "
Nov 02 13:08:13.113406 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.113263    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-token\" (UniqueName: \"kubernetes.io/projected/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-token\") pod \"eab2ec94-23e3-4528-8f27-2e9f16498268\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") "
Nov 02 13:08:13.113406 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.113337    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-workload-socket\") pod \"eab2ec94-23e3-4528-8f27-2e9f16498268\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") "
Nov 02 13:08:13.113406 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.113394    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eab2ec94-23e3-4528-8f27-2e9f16498268-kube-api-access-457qt\" (UniqueName: \"kubernetes.io/projected/eab2ec94-23e3-4528-8f27-2e9f16498268-kube-api-access-457qt\") pod \"eab2ec94-23e3-4528-8f27-2e9f16498268\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") "
Nov 02 13:08:13.113620 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.113406    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-credential-socket" pod "eab2ec94-23e3-4528-8f27-2e9f16498268" (UID: "eab2ec94-23e3-4528-8f27-2e9f16498268"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:08:13.113620 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.113435    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-workload-certs\") pod \"eab2ec94-23e3-4528-8f27-2e9f16498268\" (UID: \"eab2ec94-23e3-4528-8f27-2e9f16498268\") "
Nov 02 13:08:13.113620 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.113545    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eab2ec94-23e3-4528-8f27-2e9f16498268-istiod-ca-cert" pod "eab2ec94-23e3-4528-8f27-2e9f16498268" (UID: "eab2ec94-23e3-4528-8f27-2e9f16498268"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:08:13.113766 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.113751    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-workload-certs" pod "eab2ec94-23e3-4528-8f27-2e9f16498268" (UID: "eab2ec94-23e3-4528-8f27-2e9f16498268"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:08:13.113820 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.113784    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-workload-socket" pod "eab2ec94-23e3-4528-8f27-2e9f16498268" (UID: "eab2ec94-23e3-4528-8f27-2e9f16498268"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:08:13.113871 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.113830    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-data" pod "eab2ec94-23e3-4528-8f27-2e9f16498268" (UID: "eab2ec94-23e3-4528-8f27-2e9f16498268"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:08:13.113871 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.113835    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:08:13.113954 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.113888    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:08:13.113954 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.113905    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eab2ec94-23e3-4528-8f27-2e9f16498268-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:08:13.115463 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.115434    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-token" pod "eab2ec94-23e3-4528-8f27-2e9f16498268" (UID: "eab2ec94-23e3-4528-8f27-2e9f16498268"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:08:13.115571 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.115487    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eab2ec94-23e3-4528-8f27-2e9f16498268-kube-api-access-457qt" pod "eab2ec94-23e3-4528-8f27-2e9f16498268" (UID: "eab2ec94-23e3-4528-8f27-2e9f16498268"). InnerVolumeSpecName "kube-api-access-457qt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:08:13.115716 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.115692    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-envoy" pod "eab2ec94-23e3-4528-8f27-2e9f16498268" (UID: "eab2ec94-23e3-4528-8f27-2e9f16498268"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:08:13.115716 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.115700    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-podinfo" pod "eab2ec94-23e3-4528-8f27-2e9f16498268" (UID: "eab2ec94-23e3-4528-8f27-2e9f16498268"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 13:08:13.214828 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.214796    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-457qt\" (UniqueName: \"kubernetes.io/projected/eab2ec94-23e3-4528-8f27-2e9f16498268-kube-api-access-457qt\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:08:13.214828 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.214827    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:08:13.215029 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.214838    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:08:13.215029 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.214847    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:08:13.215029 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.214857    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eab2ec94-23e3-4528-8f27-2e9f16498268-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:08:13.215029 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.214865    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab2ec94-23e3-4528-8f27-2e9f16498268-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:08:13.988345 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.988311    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"
Nov 02 13:08:13.988345 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.988333    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv" event={"ID":"eab2ec94-23e3-4528-8f27-2e9f16498268","Type":"ContainerDied","Data":"63cb195fcc5d38317fe21bfab86d02ed6664c7b54a3987fc84249baa41420170"}
Nov 02 13:08:13.988803 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:13.988401    4942 scope.go:122] "RemoveContainer" containerID="43d8cde9fdf73327439b0c8e979145e4c9243015fb8561c177aeb7555209ff5d"
Nov 02 13:08:14.201938 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:14.201900    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"]
Nov 02 13:08:14.250783 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:14.250696    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-xhwwv"]
Nov 02 13:08:15.542349 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:15.542285    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eab2ec94-23e3-4528-8f27-2e9f16498268" path="/var/lib/kubelet/pods/eab2ec94-23e3-4528-8f27-2e9f16498268/volumes"
Nov 02 13:08:22.819284 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:22.819250    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"]
Nov 02 13:08:22.819702 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:22.819688    4942 cpu_manager.go:414] "containerMap: removing container" podUID="eab2ec94-23e3-4528-8f27-2e9f16498268" containerName="istio-proxy"
Nov 02 13:08:22.819746 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:22.819702    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eab2ec94-23e3-4528-8f27-2e9f16498268" containerName="istio-proxy"
Nov 02 13:08:22.819781 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:22.819764    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="eab2ec94-23e3-4528-8f27-2e9f16498268" containerName="istio-proxy"
Nov 02 13:08:22.822808 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:22.822790    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:22.875170 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:22.875142    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-3a78b4-openshift-default-dockercfg-84mvc\""
Nov 02 13:08:22.909027 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:22.908996    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrgbj\" (UniqueName: \"kubernetes.io/projected/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-kube-api-access-vrgbj\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:22.909157 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:22.909030    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-workload-certs\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:22.909157 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:22.909056    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-envoy\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:22.909291 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:22.909197    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istiod-ca-cert\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:22.909291 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:22.909240    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-credential-socket\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:22.909421 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:22.909373    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-podinfo\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:22.909475 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:22.909427    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-token\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:22.909534 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:22.909518    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-data\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:22.909585 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:22.909543    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-workload-socket\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:22.927494 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:22.927466    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"]
Nov 02 13:08:23.009982 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.009950    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-envoy\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.010187 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.009992    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istiod-ca-cert\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.010187 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.010010    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-credential-socket\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.010187 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.010055    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-podinfo\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.010187 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.010088    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-token\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.010187 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.010164    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-data\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.010510 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.010190    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-workload-socket\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.010510 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.010213    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrgbj\" (UniqueName: \"kubernetes.io/projected/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-kube-api-access-vrgbj\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.010510 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.010237    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-workload-certs\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.010687 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.010511    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-credential-socket\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.010807 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.010781    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-workload-socket\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.010892 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.010839    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-workload-certs\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.010892 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.010859    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istiod-ca-cert\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.010892 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.010866    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-data\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.012692 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.012665    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-envoy\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.014157 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.014135    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-podinfo\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.070990 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.070934    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-token\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.071082 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.071031    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrgbj\" (UniqueName: \"kubernetes.io/projected/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-kube-api-access-vrgbj\") pod \"e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") " pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.135867 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.135841    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:23.562763 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.562737    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"]
Nov 02 13:08:23.564177 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:08:23.564146    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc84d82aa_92f5_4c6c_bdd8_0ed6adf75c1c.slice/crio-d903199722829179abad0bc8e23b663955866ee340fe4599f1ad4898323e780d WatchSource:0}: Error finding container d903199722829179abad0bc8e23b663955866ee340fe4599f1ad4898323e780d: Status 404 returned error can't find the container with id d903199722829179abad0bc8e23b663955866ee340fe4599f1ad4898323e780d
Nov 02 13:08:23.566801 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.566769    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:08:23.566879 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.566832    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:08:23.566879 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:23.566862    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:08:24.042409 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:24.042370    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp" event={"ID":"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c","Type":"ContainerStarted","Data":"ec7909855356bb80ae1c3b4a68c67d6f0ea37ede5936a3940e51f7c3b034fad2"}
Nov 02 13:08:24.042815 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:24.042427    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp" event={"ID":"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c","Type":"ContainerStarted","Data":"d903199722829179abad0bc8e23b663955866ee340fe4599f1ad4898323e780d"}
Nov 02 13:08:24.136019 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:24.135988    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:24.137782 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:24.137751    4942 patch_prober.go:29] interesting pod/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Nov 02 13:08:24.137950 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:24.137803    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp" podUID="c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Nov 02 13:08:24.168398 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:24.166886    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp" podStartSLOduration=2.166866494 podStartE2EDuration="2.166866494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:08:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:08:24.161690268 +0000 UTC m=+4330.733040026" watchObservedRunningTime="2026-11-02 13:08:24.166866494 +0000 UTC m=+4330.738216252"
Nov 02 13:08:25.136565 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:25.136524    4942 patch_prober.go:29] interesting pod/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Nov 02 13:08:25.137000 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:25.136578    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp" podUID="c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Nov 02 13:08:26.137105 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:26.137073    4942 patch_prober.go:29] interesting pod/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Nov 02 13:08:26.137545 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:26.137136    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp" podUID="c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Nov 02 13:08:27.136254 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:27.136218    4942 patch_prober.go:29] interesting pod/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Nov 02 13:08:27.136499 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:27.136274    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp" podUID="c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Nov 02 13:08:28.136885 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:28.136852    4942 patch_prober.go:29] interesting pod/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Nov 02 13:08:28.137277 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:28.136913    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp" podUID="c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Nov 02 13:08:29.136066 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:29.136034    4942 patch_prober.go:29] interesting pod/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused" start-of-body=
Nov 02 13:08:29.136256 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:29.136081    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp" podUID="c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.118:15021/healthz/ready\": dial tcp 10.132.0.118:15021: connect: connection refused"
Nov 02 13:08:30.140181 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:30.140151    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:30.140704 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:30.140543    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:30.141221 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:08:30.141201    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:08:30.983617 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:08:30.983580    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:09:00.985517 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:09:00.985405    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:09:30.982329 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:09:30.982259    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:10:00.981976 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:10:00.981941    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:10:30.982183 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:10:30.982098    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:11:00.982688 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:11:00.982651    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:11:14.175550 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:11:14.175521    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:11:14.191803 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:11:14.191781    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:11:30.983449 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:11:30.983410    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:12:00.987433 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:12:00.987355    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:12:06.348281 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:06.348246    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj"]
Nov 02 13:12:06.348866 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:06.348781    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj" podUID="3e476d3d-1090-4701-8a38-fd60c3f0d715" containerName="sim" containerID="cri-o://3c2d74a2600ef002c9cd89a2a55cfd93111a86327d3fd8ae7c3052d0c592ef39" gracePeriod=30
Nov 02 13:12:06.600260 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:06.600235    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj"
Nov 02 13:12:06.713262 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:06.713227    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e476d3d-1090-4701-8a38-fd60c3f0d715-kube-api-access-9p9zs\" (UniqueName: \"kubernetes.io/projected/3e476d3d-1090-4701-8a38-fd60c3f0d715-kube-api-access-9p9zs\") pod \"3e476d3d-1090-4701-8a38-fd60c3f0d715\" (UID: \"3e476d3d-1090-4701-8a38-fd60c3f0d715\") "
Nov 02 13:12:06.713876 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:06.713377    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3e476d3d-1090-4701-8a38-fd60c3f0d715-tls\" (UniqueName: \"kubernetes.io/secret/3e476d3d-1090-4701-8a38-fd60c3f0d715-tls\") pod \"3e476d3d-1090-4701-8a38-fd60c3f0d715\" (UID: \"3e476d3d-1090-4701-8a38-fd60c3f0d715\") "
Nov 02 13:12:06.716052 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:06.716020    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e476d3d-1090-4701-8a38-fd60c3f0d715-tls" pod "3e476d3d-1090-4701-8a38-fd60c3f0d715" (UID: "3e476d3d-1090-4701-8a38-fd60c3f0d715"). InnerVolumeSpecName "tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:12:06.716155 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:06.716088    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e476d3d-1090-4701-8a38-fd60c3f0d715-kube-api-access-9p9zs" pod "3e476d3d-1090-4701-8a38-fd60c3f0d715" (UID: "3e476d3d-1090-4701-8a38-fd60c3f0d715"). InnerVolumeSpecName "kube-api-access-9p9zs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:12:06.815134 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:06.815098    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9p9zs\" (UniqueName: \"kubernetes.io/projected/3e476d3d-1090-4701-8a38-fd60c3f0d715-kube-api-access-9p9zs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:12:06.815134 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:06.815128    4942 reconciler_common.go:299] "Volume detached for volume \"tls\" (UniqueName: \"kubernetes.io/secret/3e476d3d-1090-4701-8a38-fd60c3f0d715-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:12:07.262568 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:07.262533    4942 generic.go:356] "Generic (PLEG): container finished" podID="3e476d3d-1090-4701-8a38-fd60c3f0d715" containerID="3c2d74a2600ef002c9cd89a2a55cfd93111a86327d3fd8ae7c3052d0c592ef39" exitCode=0
Nov 02 13:12:07.262824 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:07.262805    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj" event={"ID":"3e476d3d-1090-4701-8a38-fd60c3f0d715","Type":"ContainerDied","Data":"3c2d74a2600ef002c9cd89a2a55cfd93111a86327d3fd8ae7c3052d0c592ef39"}
Nov 02 13:12:07.262940 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:07.262924    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj" event={"ID":"3e476d3d-1090-4701-8a38-fd60c3f0d715","Type":"ContainerDied","Data":"f87fa63158c8356bb92662489040f83f490b98be5b2870dbdc394e0763f76036"}
Nov 02 13:12:07.263040 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:07.263028    4942 scope.go:122] "RemoveContainer" containerID="3c2d74a2600ef002c9cd89a2a55cfd93111a86327d3fd8ae7c3052d0c592ef39"
Nov 02 13:12:07.263443 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:07.263251    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj"
Nov 02 13:12:07.275258 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:07.275236    4942 scope.go:122] "RemoveContainer" containerID="3c2d74a2600ef002c9cd89a2a55cfd93111a86327d3fd8ae7c3052d0c592ef39"
Nov 02 13:12:07.275554 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:12:07.275526    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c2d74a2600ef002c9cd89a2a55cfd93111a86327d3fd8ae7c3052d0c592ef39\": container with ID starting with 3c2d74a2600ef002c9cd89a2a55cfd93111a86327d3fd8ae7c3052d0c592ef39 not found: ID does not exist" containerID="3c2d74a2600ef002c9cd89a2a55cfd93111a86327d3fd8ae7c3052d0c592ef39"
Nov 02 13:12:07.275611 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:07.275555    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c2d74a2600ef002c9cd89a2a55cfd93111a86327d3fd8ae7c3052d0c592ef39"} err="failed to get container status \"3c2d74a2600ef002c9cd89a2a55cfd93111a86327d3fd8ae7c3052d0c592ef39\": rpc error: code = NotFound desc = could not find container \"3c2d74a2600ef002c9cd89a2a55cfd93111a86327d3fd8ae7c3052d0c592ef39\": container with ID starting with 3c2d74a2600ef002c9cd89a2a55cfd93111a86327d3fd8ae7c3052d0c592ef39 not found: ID does not exist"
Nov 02 13:12:07.426129 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:07.426095    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj"]
Nov 02 13:12:07.475480 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:07.475443    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-extmod-shr-user-1125bd/e2e-usage-sim-eb01cf-558676877d-8tvmj"]
Nov 02 13:12:07.542578 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:12:07.542510    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3e476d3d-1090-4701-8a38-fd60c3f0d715" path="/var/lib/kubelet/pods/3e476d3d-1090-4701-8a38-fd60c3f0d715/volumes"
Nov 02 13:12:30.982932 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:12:30.982893    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:13:00.985428 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:13:00.985390    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:13:14.678521 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:14.678487    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5"]
Nov 02 13:13:14.788899 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:14.788863    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm"]
Nov 02 13:13:18.587676 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:18.587639    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"]
Nov 02 13:13:18.588091 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:18.587854    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz" podUID="3cf8ac58-6048-4b60-8f6a-487dde6cd593" containerName="maas-api" containerID="cri-o://9bfab9066dafa14a4ea2e57059aef1b3763d487a1710d46b8e8487429a69ba8e" gracePeriod=30
Nov 02 13:13:18.736824 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:18.736782    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz" podUID="3cf8ac58-6048-4b60-8f6a-487dde6cd593" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.85:8443/health\": dial tcp 10.132.0.85:8443: connect: connection refused"
Nov 02 13:13:18.838852 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:18.838795    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"
Nov 02 13:13:18.887630 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:18.887598    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3cf8ac58-6048-4b60-8f6a-487dde6cd593-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cf8ac58-6048-4b60-8f6a-487dde6cd593-maas-api-tls\") pod \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\" (UID: \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\") "
Nov 02 13:13:18.887783 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:18.887664    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3cf8ac58-6048-4b60-8f6a-487dde6cd593-kube-api-access-mrlvl\" (UniqueName: \"kubernetes.io/projected/3cf8ac58-6048-4b60-8f6a-487dde6cd593-kube-api-access-mrlvl\") pod \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\" (UID: \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\") "
Nov 02 13:13:18.887783 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:18.887770    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3cf8ac58-6048-4b60-8f6a-487dde6cd593-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf8ac58-6048-4b60-8f6a-487dde6cd593-openshift-service-ca\") pod \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\" (UID: \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\") "
Nov 02 13:13:18.887905 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:18.887822    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3cf8ac58-6048-4b60-8f6a-487dde6cd593-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cf8ac58-6048-4b60-8f6a-487dde6cd593-metrics-certs\") pod \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\" (UID: \"3cf8ac58-6048-4b60-8f6a-487dde6cd593\") "
Nov 02 13:13:18.888174 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:18.888150    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cf8ac58-6048-4b60-8f6a-487dde6cd593-openshift-service-ca" pod "3cf8ac58-6048-4b60-8f6a-487dde6cd593" (UID: "3cf8ac58-6048-4b60-8f6a-487dde6cd593"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:13:18.889790 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:18.889762    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf8ac58-6048-4b60-8f6a-487dde6cd593-metrics-certs" pod "3cf8ac58-6048-4b60-8f6a-487dde6cd593" (UID: "3cf8ac58-6048-4b60-8f6a-487dde6cd593"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:13:18.889790 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:18.889767    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cf8ac58-6048-4b60-8f6a-487dde6cd593-kube-api-access-mrlvl" pod "3cf8ac58-6048-4b60-8f6a-487dde6cd593" (UID: "3cf8ac58-6048-4b60-8f6a-487dde6cd593"). InnerVolumeSpecName "kube-api-access-mrlvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:13:18.889925 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:18.889836    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf8ac58-6048-4b60-8f6a-487dde6cd593-maas-api-tls" pod "3cf8ac58-6048-4b60-8f6a-487dde6cd593" (UID: "3cf8ac58-6048-4b60-8f6a-487dde6cd593"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:13:18.989719 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:18.989676    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cf8ac58-6048-4b60-8f6a-487dde6cd593-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:13:18.989881 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:18.989723    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrlvl\" (UniqueName: \"kubernetes.io/projected/3cf8ac58-6048-4b60-8f6a-487dde6cd593-kube-api-access-mrlvl\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:13:18.989881 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:18.989742    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf8ac58-6048-4b60-8f6a-487dde6cd593-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:13:18.989881 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:18.989757    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cf8ac58-6048-4b60-8f6a-487dde6cd593-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:13:19.660420 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:19.660383    4942 generic.go:356] "Generic (PLEG): container finished" podID="3cf8ac58-6048-4b60-8f6a-487dde6cd593" containerID="9bfab9066dafa14a4ea2e57059aef1b3763d487a1710d46b8e8487429a69ba8e" exitCode=0
Nov 02 13:13:19.660866 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:19.660459    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"
Nov 02 13:13:19.660866 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:19.660477    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz" event={"ID":"3cf8ac58-6048-4b60-8f6a-487dde6cd593","Type":"ContainerDied","Data":"9bfab9066dafa14a4ea2e57059aef1b3763d487a1710d46b8e8487429a69ba8e"}
Nov 02 13:13:19.660866 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:19.660532    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz" event={"ID":"3cf8ac58-6048-4b60-8f6a-487dde6cd593","Type":"ContainerDied","Data":"89f2a3ecbe69df24b81a514a7b56f46461840d65d7f8335fe6001cb427a8a67c"}
Nov 02 13:13:19.660866 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:19.660555    4942 scope.go:122] "RemoveContainer" containerID="9bfab9066dafa14a4ea2e57059aef1b3763d487a1710d46b8e8487429a69ba8e"
Nov 02 13:13:19.670836 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:19.670819    4942 scope.go:122] "RemoveContainer" containerID="9bfab9066dafa14a4ea2e57059aef1b3763d487a1710d46b8e8487429a69ba8e"
Nov 02 13:13:19.671125 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:13:19.671107    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bfab9066dafa14a4ea2e57059aef1b3763d487a1710d46b8e8487429a69ba8e\": container with ID starting with 9bfab9066dafa14a4ea2e57059aef1b3763d487a1710d46b8e8487429a69ba8e not found: ID does not exist" containerID="9bfab9066dafa14a4ea2e57059aef1b3763d487a1710d46b8e8487429a69ba8e"
Nov 02 13:13:19.671193 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:19.671140    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bfab9066dafa14a4ea2e57059aef1b3763d487a1710d46b8e8487429a69ba8e"} err="failed to get container status \"9bfab9066dafa14a4ea2e57059aef1b3763d487a1710d46b8e8487429a69ba8e\": rpc error: code = NotFound desc = could not find container \"9bfab9066dafa14a4ea2e57059aef1b3763d487a1710d46b8e8487429a69ba8e\": container with ID starting with 9bfab9066dafa14a4ea2e57059aef1b3763d487a1710d46b8e8487429a69ba8e not found: ID does not exist"
Nov 02 13:13:19.679263 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:19.679231    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5" podUID="36599f9c-1da2-4ecf-a1ca-78382bc3ad9b" containerName="payload-processing" containerID="cri-o://d4fd307346542846e73bd33905803e891e1cfa52cb4837dcfca377a9966d6f52" gracePeriod=25
Nov 02 13:13:19.789428 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:19.789393    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm" podUID="6857980e-39b1-408b-9f98-71c8ec07677e" containerName="payload-pre-processing" containerID="cri-o://ad41dfe2d0231cde8602ab736e52a78af41d9600632777df7c3c9eea61b2ffde" gracePeriod=25
Nov 02 13:13:19.824512 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:19.824450    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"]
Nov 02 13:13:19.828456 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:19.828433    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6ac03f-d8ccb8c59-cnbgz"]
Nov 02 13:13:20.803372 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:20.803336    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5" podUID="36599f9c-1da2-4ecf-a1ca-78382bc3ad9b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 13:13:20.803752 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:20.803348    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm" podUID="6857980e-39b1-408b-9f98-71c8ec07677e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 13:13:21.541703 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:21.541663    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3cf8ac58-6048-4b60-8f6a-487dde6cd593" path="/var/lib/kubelet/pods/3cf8ac58-6048-4b60-8f6a-487dde6cd593/volumes"
Nov 02 13:13:30.803772 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:30.803681    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5" podUID="36599f9c-1da2-4ecf-a1ca-78382bc3ad9b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 13:13:30.803772 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:30.803732    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm" podUID="6857980e-39b1-408b-9f98-71c8ec07677e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 13:13:30.982882 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:13:30.982848    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:13:39.934767 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:39.934703    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5"
Nov 02 13:13:40.035044 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.035015    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm"
Nov 02 13:13:40.083360 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.083328    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36599f9c-1da2-4ecf-a1ca-78382bc3ad9b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36599f9c-1da2-4ecf-a1ca-78382bc3ad9b-plugins-config-volume\") pod \"36599f9c-1da2-4ecf-a1ca-78382bc3ad9b\" (UID: \"36599f9c-1da2-4ecf-a1ca-78382bc3ad9b\") "
Nov 02 13:13:40.083491 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.083422    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36599f9c-1da2-4ecf-a1ca-78382bc3ad9b-kube-api-access-h9jgh\" (UniqueName: \"kubernetes.io/projected/36599f9c-1da2-4ecf-a1ca-78382bc3ad9b-kube-api-access-h9jgh\") pod \"36599f9c-1da2-4ecf-a1ca-78382bc3ad9b\" (UID: \"36599f9c-1da2-4ecf-a1ca-78382bc3ad9b\") "
Nov 02 13:13:40.083729 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.083703    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36599f9c-1da2-4ecf-a1ca-78382bc3ad9b-plugins-config-volume" pod "36599f9c-1da2-4ecf-a1ca-78382bc3ad9b" (UID: "36599f9c-1da2-4ecf-a1ca-78382bc3ad9b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:13:40.085383 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.085359    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36599f9c-1da2-4ecf-a1ca-78382bc3ad9b-kube-api-access-h9jgh" pod "36599f9c-1da2-4ecf-a1ca-78382bc3ad9b" (UID: "36599f9c-1da2-4ecf-a1ca-78382bc3ad9b"). InnerVolumeSpecName "kube-api-access-h9jgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:13:40.184601 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.184575    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6857980e-39b1-408b-9f98-71c8ec07677e-kube-api-access-f4x4m\" (UniqueName: \"kubernetes.io/projected/6857980e-39b1-408b-9f98-71c8ec07677e-kube-api-access-f4x4m\") pod \"6857980e-39b1-408b-9f98-71c8ec07677e\" (UID: \"6857980e-39b1-408b-9f98-71c8ec07677e\") "
Nov 02 13:13:40.184800 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.184683    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6857980e-39b1-408b-9f98-71c8ec07677e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6857980e-39b1-408b-9f98-71c8ec07677e-plugins-config-volume\") pod \"6857980e-39b1-408b-9f98-71c8ec07677e\" (UID: \"6857980e-39b1-408b-9f98-71c8ec07677e\") "
Nov 02 13:13:40.184889 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.184847    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9jgh\" (UniqueName: \"kubernetes.io/projected/36599f9c-1da2-4ecf-a1ca-78382bc3ad9b-kube-api-access-h9jgh\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:13:40.184889 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.184859    4942 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/36599f9c-1da2-4ecf-a1ca-78382bc3ad9b-plugins-config-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:13:40.185057 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.185036    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6857980e-39b1-408b-9f98-71c8ec07677e-plugins-config-volume" pod "6857980e-39b1-408b-9f98-71c8ec07677e" (UID: "6857980e-39b1-408b-9f98-71c8ec07677e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:13:40.186679 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.186658    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6857980e-39b1-408b-9f98-71c8ec07677e-kube-api-access-f4x4m" pod "6857980e-39b1-408b-9f98-71c8ec07677e" (UID: "6857980e-39b1-408b-9f98-71c8ec07677e"). InnerVolumeSpecName "kube-api-access-f4x4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:13:40.285992 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.285962    4942 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6857980e-39b1-408b-9f98-71c8ec07677e-plugins-config-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:13:40.285992 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.285988    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4x4m\" (UniqueName: \"kubernetes.io/projected/6857980e-39b1-408b-9f98-71c8ec07677e-kube-api-access-f4x4m\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:13:40.772826 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.772790    4942 generic.go:356] "Generic (PLEG): container finished" podID="36599f9c-1da2-4ecf-a1ca-78382bc3ad9b" containerID="d4fd307346542846e73bd33905803e891e1cfa52cb4837dcfca377a9966d6f52" exitCode=0
Nov 02 13:13:40.773033 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.772843    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5"
Nov 02 13:13:40.773033 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.772876    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5" event={"ID":"36599f9c-1da2-4ecf-a1ca-78382bc3ad9b","Type":"ContainerDied","Data":"d4fd307346542846e73bd33905803e891e1cfa52cb4837dcfca377a9966d6f52"}
Nov 02 13:13:40.773033 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.772926    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5" event={"ID":"36599f9c-1da2-4ecf-a1ca-78382bc3ad9b","Type":"ContainerDied","Data":"b3e199e4f2c03ace773ded9a815d765c131d308487df5a4a2daa11aacc651ef2"}
Nov 02 13:13:40.773033 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.772948    4942 scope.go:122] "RemoveContainer" containerID="d4fd307346542846e73bd33905803e891e1cfa52cb4837dcfca377a9966d6f52"
Nov 02 13:13:40.774380 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.774354    4942 generic.go:356] "Generic (PLEG): container finished" podID="6857980e-39b1-408b-9f98-71c8ec07677e" containerID="ad41dfe2d0231cde8602ab736e52a78af41d9600632777df7c3c9eea61b2ffde" exitCode=0
Nov 02 13:13:40.774517 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.774405    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm"
Nov 02 13:13:40.774517 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.774437    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm" event={"ID":"6857980e-39b1-408b-9f98-71c8ec07677e","Type":"ContainerDied","Data":"ad41dfe2d0231cde8602ab736e52a78af41d9600632777df7c3c9eea61b2ffde"}
Nov 02 13:13:40.774517 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.774486    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm" event={"ID":"6857980e-39b1-408b-9f98-71c8ec07677e","Type":"ContainerDied","Data":"1dd719c89777dd06bc41daef31707952c0feacfaea98d936b72250eb669e33ed"}
Nov 02 13:13:40.787512 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.787488    4942 scope.go:122] "RemoveContainer" containerID="d4fd307346542846e73bd33905803e891e1cfa52cb4837dcfca377a9966d6f52"
Nov 02 13:13:40.787998 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:13:40.787976    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4fd307346542846e73bd33905803e891e1cfa52cb4837dcfca377a9966d6f52\": container with ID starting with d4fd307346542846e73bd33905803e891e1cfa52cb4837dcfca377a9966d6f52 not found: ID does not exist" containerID="d4fd307346542846e73bd33905803e891e1cfa52cb4837dcfca377a9966d6f52"
Nov 02 13:13:40.788065 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.788007    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4fd307346542846e73bd33905803e891e1cfa52cb4837dcfca377a9966d6f52"} err="failed to get container status \"d4fd307346542846e73bd33905803e891e1cfa52cb4837dcfca377a9966d6f52\": rpc error: code = NotFound desc = could not find container \"d4fd307346542846e73bd33905803e891e1cfa52cb4837dcfca377a9966d6f52\": container with ID starting with d4fd307346542846e73bd33905803e891e1cfa52cb4837dcfca377a9966d6f52 not found: ID does not exist"
Nov 02 13:13:40.788065 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.788024    4942 scope.go:122] "RemoveContainer" containerID="ad41dfe2d0231cde8602ab736e52a78af41d9600632777df7c3c9eea61b2ffde"
Nov 02 13:13:40.796159 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.795964    4942 scope.go:122] "RemoveContainer" containerID="ad41dfe2d0231cde8602ab736e52a78af41d9600632777df7c3c9eea61b2ffde"
Nov 02 13:13:40.796232 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:13:40.796215    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad41dfe2d0231cde8602ab736e52a78af41d9600632777df7c3c9eea61b2ffde\": container with ID starting with ad41dfe2d0231cde8602ab736e52a78af41d9600632777df7c3c9eea61b2ffde not found: ID does not exist" containerID="ad41dfe2d0231cde8602ab736e52a78af41d9600632777df7c3c9eea61b2ffde"
Nov 02 13:13:40.796283 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.796238    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad41dfe2d0231cde8602ab736e52a78af41d9600632777df7c3c9eea61b2ffde"} err="failed to get container status \"ad41dfe2d0231cde8602ab736e52a78af41d9600632777df7c3c9eea61b2ffde\": rpc error: code = NotFound desc = could not find container \"ad41dfe2d0231cde8602ab736e52a78af41d9600632777df7c3c9eea61b2ffde\": container with ID starting with ad41dfe2d0231cde8602ab736e52a78af41d9600632777df7c3c9eea61b2ffde not found: ID does not exist"
Nov 02 13:13:40.936436 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.936408    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm"]
Nov 02 13:13:40.940842 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:40.940813    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6ac03f-69955c8999-lgtmm"]
Nov 02 13:13:41.049545 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:41.049469    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5"]
Nov 02 13:13:41.053452 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:41.053426    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-6ac03f-75c4d699fc-822x5"]
Nov 02 13:13:41.543450 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:41.543421    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36599f9c-1da2-4ecf-a1ca-78382bc3ad9b" path="/var/lib/kubelet/pods/36599f9c-1da2-4ecf-a1ca-78382bc3ad9b/volumes"
Nov 02 13:13:41.543790 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:41.543778    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6857980e-39b1-408b-9f98-71c8ec07677e" path="/var/lib/kubelet/pods/6857980e-39b1-408b-9f98-71c8ec07677e/volumes"
Nov 02 13:13:49.245712 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:49.245680    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"]
Nov 02 13:13:49.246194 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:49.245873    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp" podUID="c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" containerName="istio-proxy" containerID="cri-o://ec7909855356bb80ae1c3b4a68c67d6f0ea37ede5936a3940e51f7c3b034fad2" gracePeriod=30
Nov 02 13:13:50.334341 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:50.334275    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-fb96bcc76-d9wdp_a8304b1f-5415-4992-a8d5-23e2c27e6b28/authorino/0.log"
Nov 02 13:13:51.073406 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:51.073374    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-56ffx_58467a64-e86d-42d0-9ef7-8fcafca6a9e5/manager/0.log"
Nov 02 13:13:52.351615 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:52.351581    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-7wrsc_fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902/manager/0.log"
Nov 02 13:13:52.990021 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:52.989978    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-vktcs_da858d4c-2fba-41ef-a18a-2ee3d0dcbbea/manager/0.log"
Nov 02 13:13:54.475276 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.475251    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-t5hm9_3e1f269e-665c-4850-a92e-3fc1604c6efb/manager/0.log"
Nov 02 13:13:54.494125 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.494105    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:13:54.512448 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.512401    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-token\" (UniqueName: \"kubernetes.io/projected/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-token\") pod \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") "
Nov 02 13:13:54.512448 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.512437    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-workload-certs\") pod \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") "
Nov 02 13:13:54.512605 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.512459    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-envoy\") pod \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") "
Nov 02 13:13:54.512605 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.512514    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istiod-ca-cert\") pod \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") "
Nov 02 13:13:54.512605 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.512581    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-data\") pod \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") "
Nov 02 13:13:54.512757 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.512609    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-credential-socket\") pod \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") "
Nov 02 13:13:54.512757 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.512632    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-kube-api-access-vrgbj\" (UniqueName: \"kubernetes.io/projected/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-kube-api-access-vrgbj\") pod \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") "
Nov 02 13:13:54.512757 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.512674    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-podinfo\") pod \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") "
Nov 02 13:13:54.512757 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.512672    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-workload-certs" pod "c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" (UID: "c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:13:54.512757 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.512701    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-workload-socket\") pod \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\" (UID: \"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c\") "
Nov 02 13:13:54.513107 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.513066    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-workload-socket" pod "c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" (UID: "c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:13:54.513107 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.513050    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-credential-socket" pod "c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" (UID: "c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:13:54.513107 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.513091    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istiod-ca-cert" pod "c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" (UID: "c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:13:54.513285 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.513215    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-data" pod "c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" (UID: "c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:13:54.513388 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.513362    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:13:54.513452 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.513394    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:13:54.513452 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.513411    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:13:54.513452 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.513426    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:13:54.513452 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.513440    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:13:54.514701 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.514672    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-token" pod "c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" (UID: "c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:13:54.514802 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.514740    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-envoy" pod "c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" (UID: "c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:13:54.514802 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.514787    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-podinfo" pod "c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" (UID: "c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 13:13:54.514903 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.514836    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-kube-api-access-vrgbj" pod "c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" (UID: "c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c"). InnerVolumeSpecName "kube-api-access-vrgbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:13:54.614837 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.614804    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrgbj\" (UniqueName: \"kubernetes.io/projected/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-kube-api-access-vrgbj\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:13:54.614837 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.614830    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:13:54.614999 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.614846    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:13:54.614999 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.614858    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:13:54.855723 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.855688    4942 generic.go:356] "Generic (PLEG): container finished" podID="c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" containerID="ec7909855356bb80ae1c3b4a68c67d6f0ea37ede5936a3940e51f7c3b034fad2" exitCode=0
Nov 02 13:13:54.855917 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.855736    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"
Nov 02 13:13:54.855917 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.855767    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp" event={"ID":"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c","Type":"ContainerDied","Data":"ec7909855356bb80ae1c3b4a68c67d6f0ea37ede5936a3940e51f7c3b034fad2"}
Nov 02 13:13:54.855917 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.855826    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp" event={"ID":"c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c","Type":"ContainerDied","Data":"d903199722829179abad0bc8e23b663955866ee340fe4599f1ad4898323e780d"}
Nov 02 13:13:54.855917 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.855841    4942 scope.go:122] "RemoveContainer" containerID="ec7909855356bb80ae1c3b4a68c67d6f0ea37ede5936a3940e51f7c3b034fad2"
Nov 02 13:13:54.865976 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.865959    4942 scope.go:122] "RemoveContainer" containerID="ec7909855356bb80ae1c3b4a68c67d6f0ea37ede5936a3940e51f7c3b034fad2"
Nov 02 13:13:54.866240 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:13:54.866223    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec7909855356bb80ae1c3b4a68c67d6f0ea37ede5936a3940e51f7c3b034fad2\": container with ID starting with ec7909855356bb80ae1c3b4a68c67d6f0ea37ede5936a3940e51f7c3b034fad2 not found: ID does not exist" containerID="ec7909855356bb80ae1c3b4a68c67d6f0ea37ede5936a3940e51f7c3b034fad2"
Nov 02 13:13:54.866283 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:54.866248    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec7909855356bb80ae1c3b4a68c67d6f0ea37ede5936a3940e51f7c3b034fad2"} err="failed to get container status \"ec7909855356bb80ae1c3b4a68c67d6f0ea37ede5936a3940e51f7c3b034fad2\": rpc error: code = NotFound desc = could not find container \"ec7909855356bb80ae1c3b4a68c67d6f0ea37ede5936a3940e51f7c3b034fad2\": container with ID starting with ec7909855356bb80ae1c3b4a68c67d6f0ea37ede5936a3940e51f7c3b034fad2 not found: ID does not exist"
Nov 02 13:13:55.020798 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:55.020768    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"]
Nov 02 13:13:55.024664 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:55.024640    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-3a78b4-openshift-default-77b8648d45-z9lxp"]
Nov 02 13:13:55.542692 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:13:55.542661    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" path="/var/lib/kubelet/pods/c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c/volumes"
Nov 02 13:14:00.982062 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:14:00.982026    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:14:11.228229 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.228192    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"]
Nov 02 13:14:11.228625 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.228581    4942 cpu_manager.go:414] "containerMap: removing container" podUID="6857980e-39b1-408b-9f98-71c8ec07677e" containerName="payload-pre-processing"
Nov 02 13:14:11.228625 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.228594    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6857980e-39b1-408b-9f98-71c8ec07677e" containerName="payload-pre-processing"
Nov 02 13:14:11.228625 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.228610    4942 cpu_manager.go:414] "containerMap: removing container" podUID="36599f9c-1da2-4ecf-a1ca-78382bc3ad9b" containerName="payload-processing"
Nov 02 13:14:11.228625 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.228619    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36599f9c-1da2-4ecf-a1ca-78382bc3ad9b" containerName="payload-processing"
Nov 02 13:14:11.228760 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.228639    4942 cpu_manager.go:414] "containerMap: removing container" podUID="c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" containerName="istio-proxy"
Nov 02 13:14:11.228760 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.228647    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" containerName="istio-proxy"
Nov 02 13:14:11.228760 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.228676    4942 cpu_manager.go:414] "containerMap: removing container" podUID="3cf8ac58-6048-4b60-8f6a-487dde6cd593" containerName="maas-api"
Nov 02 13:14:11.228760 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.228684    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3cf8ac58-6048-4b60-8f6a-487dde6cd593" containerName="maas-api"
Nov 02 13:14:11.228760 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.228697    4942 cpu_manager.go:414] "containerMap: removing container" podUID="3e476d3d-1090-4701-8a38-fd60c3f0d715" containerName="sim"
Nov 02 13:14:11.228760 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.228723    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e476d3d-1090-4701-8a38-fd60c3f0d715" containerName="sim"
Nov 02 13:14:11.229323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.228838    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="3cf8ac58-6048-4b60-8f6a-487dde6cd593" containerName="maas-api"
Nov 02 13:14:11.229323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.228859    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e476d3d-1090-4701-8a38-fd60c3f0d715" containerName="sim"
Nov 02 13:14:11.229323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.228869    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="6857980e-39b1-408b-9f98-71c8ec07677e" containerName="payload-pre-processing"
Nov 02 13:14:11.229323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.228876    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="36599f9c-1da2-4ecf-a1ca-78382bc3ad9b" containerName="payload-processing"
Nov 02 13:14:11.229323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.228884    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="c84d82aa-92f5-4c6c-bdd8-0ed6adf75c1c" containerName="istio-proxy"
Nov 02 13:14:11.233341 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.233323    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"
Nov 02 13:14:11.337682 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.337651    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c1934578-serving-cert\""
Nov 02 13:14:11.337824 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.337762    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pl94d\""
Nov 02 13:14:11.337824 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.337788    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 02 13:14:11.340219 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.340195    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"]
Nov 02 13:14:11.366523 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.366495    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e44ee83-d3ce-4dee-b442-c1a712027d9e-openshift-service-ca\") pod \"maas-api-e2e-ait-c1934578-67c9dd9857-vprj2\" (UID: \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"
Nov 02 13:14:11.366646 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.366557    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p44t6\" (UniqueName: \"kubernetes.io/projected/0e44ee83-d3ce-4dee-b442-c1a712027d9e-kube-api-access-p44t6\") pod \"maas-api-e2e-ait-c1934578-67c9dd9857-vprj2\" (UID: \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"
Nov 02 13:14:11.366702 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.366671    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e44ee83-d3ce-4dee-b442-c1a712027d9e-maas-api-tls\") pod \"maas-api-e2e-ait-c1934578-67c9dd9857-vprj2\" (UID: \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"
Nov 02 13:14:11.366750 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.366714    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e44ee83-d3ce-4dee-b442-c1a712027d9e-metrics-certs\") pod \"maas-api-e2e-ait-c1934578-67c9dd9857-vprj2\" (UID: \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"
Nov 02 13:14:11.467643 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.467615    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p44t6\" (UniqueName: \"kubernetes.io/projected/0e44ee83-d3ce-4dee-b442-c1a712027d9e-kube-api-access-p44t6\") pod \"maas-api-e2e-ait-c1934578-67c9dd9857-vprj2\" (UID: \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"
Nov 02 13:14:11.467780 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.467653    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e44ee83-d3ce-4dee-b442-c1a712027d9e-maas-api-tls\") pod \"maas-api-e2e-ait-c1934578-67c9dd9857-vprj2\" (UID: \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"
Nov 02 13:14:11.467780 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.467678    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e44ee83-d3ce-4dee-b442-c1a712027d9e-metrics-certs\") pod \"maas-api-e2e-ait-c1934578-67c9dd9857-vprj2\" (UID: \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"
Nov 02 13:14:11.467780 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.467763    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e44ee83-d3ce-4dee-b442-c1a712027d9e-openshift-service-ca\") pod \"maas-api-e2e-ait-c1934578-67c9dd9857-vprj2\" (UID: \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"
Nov 02 13:14:11.468439 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.468413    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e44ee83-d3ce-4dee-b442-c1a712027d9e-openshift-service-ca\") pod \"maas-api-e2e-ait-c1934578-67c9dd9857-vprj2\" (UID: \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"
Nov 02 13:14:11.470150 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.470130    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e44ee83-d3ce-4dee-b442-c1a712027d9e-metrics-certs\") pod \"maas-api-e2e-ait-c1934578-67c9dd9857-vprj2\" (UID: \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"
Nov 02 13:14:11.470222 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.470174    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e44ee83-d3ce-4dee-b442-c1a712027d9e-maas-api-tls\") pod \"maas-api-e2e-ait-c1934578-67c9dd9857-vprj2\" (UID: \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"
Nov 02 13:14:11.530841 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.530781    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p44t6\" (UniqueName: \"kubernetes.io/projected/0e44ee83-d3ce-4dee-b442-c1a712027d9e-kube-api-access-p44t6\") pod \"maas-api-e2e-ait-c1934578-67c9dd9857-vprj2\" (UID: \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"
Nov 02 13:14:11.546064 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.546039    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"
Nov 02 13:14:11.921872 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.921849    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"]
Nov 02 13:14:11.923922 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:14:11.923896    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e44ee83_d3ce_4dee_b442_c1a712027d9e.slice/crio-f58b649b3fd91b324ce2d6bb033908d79c3b1e861db5f86c437245ce92e6f56b WatchSource:0}: Error finding container f58b649b3fd91b324ce2d6bb033908d79c3b1e861db5f86c437245ce92e6f56b: Status 404 returned error can't find the container with id f58b649b3fd91b324ce2d6bb033908d79c3b1e861db5f86c437245ce92e6f56b
Nov 02 13:14:11.925244 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.925216    4942 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 13:14:11.945146 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:11.945116    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2" event={"ID":"0e44ee83-d3ce-4dee-b442-c1a712027d9e","Type":"ContainerStarted","Data":"f58b649b3fd91b324ce2d6bb033908d79c3b1e861db5f86c437245ce92e6f56b"}
Nov 02 13:14:13.958140 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:13.958104    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2" event={"ID":"0e44ee83-d3ce-4dee-b442-c1a712027d9e","Type":"ContainerStarted","Data":"51cd292b0d2c8411c3084877c132d04c5dac7a23ccfe7f757f28ed5f25a937df"}
Nov 02 13:14:13.958566 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:13.958193    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"
Nov 02 13:14:14.076340 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:14.076266    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2" podStartSLOduration=2.103160589 podStartE2EDuration="3.076250286s" totalImagesPullingTime="973.089697ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:14:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:14:14.07523135 +0000 UTC m=+4680.646581107" watchObservedRunningTime="2026-11-02 13:14:14.076250286 +0000 UTC m=+4680.647600040"
Nov 02 13:14:19.969595 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:14:19.969561    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"
Nov 02 13:14:30.982734 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:14:30.982696    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:15:00.359191 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:00.359113    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"]
Nov 02 13:15:00.359612 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:00.359334    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2" podUID="0e44ee83-d3ce-4dee-b442-c1a712027d9e" containerName="maas-api" containerID="cri-o://51cd292b0d2c8411c3084877c132d04c5dac7a23ccfe7f757f28ed5f25a937df" gracePeriod=30
Nov 02 13:15:00.606514 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:00.606491    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"
Nov 02 13:15:00.704933 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:00.704867    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e44ee83-d3ce-4dee-b442-c1a712027d9e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e44ee83-d3ce-4dee-b442-c1a712027d9e-metrics-certs\") pod \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\" (UID: \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\") "
Nov 02 13:15:00.704933 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:00.704908    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e44ee83-d3ce-4dee-b442-c1a712027d9e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e44ee83-d3ce-4dee-b442-c1a712027d9e-openshift-service-ca\") pod \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\" (UID: \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\") "
Nov 02 13:15:00.705119 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:00.704985    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e44ee83-d3ce-4dee-b442-c1a712027d9e-kube-api-access-p44t6\" (UniqueName: \"kubernetes.io/projected/0e44ee83-d3ce-4dee-b442-c1a712027d9e-kube-api-access-p44t6\") pod \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\" (UID: \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\") "
Nov 02 13:15:00.705119 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:00.705023    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e44ee83-d3ce-4dee-b442-c1a712027d9e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e44ee83-d3ce-4dee-b442-c1a712027d9e-maas-api-tls\") pod \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\" (UID: \"0e44ee83-d3ce-4dee-b442-c1a712027d9e\") "
Nov 02 13:15:00.705370 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:00.705341    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e44ee83-d3ce-4dee-b442-c1a712027d9e-openshift-service-ca" pod "0e44ee83-d3ce-4dee-b442-c1a712027d9e" (UID: "0e44ee83-d3ce-4dee-b442-c1a712027d9e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:15:00.706839 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:00.706812    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e44ee83-d3ce-4dee-b442-c1a712027d9e-metrics-certs" pod "0e44ee83-d3ce-4dee-b442-c1a712027d9e" (UID: "0e44ee83-d3ce-4dee-b442-c1a712027d9e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:15:00.706966 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:00.706951    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e44ee83-d3ce-4dee-b442-c1a712027d9e-maas-api-tls" pod "0e44ee83-d3ce-4dee-b442-c1a712027d9e" (UID: "0e44ee83-d3ce-4dee-b442-c1a712027d9e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:15:00.707038 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:00.707023    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e44ee83-d3ce-4dee-b442-c1a712027d9e-kube-api-access-p44t6" pod "0e44ee83-d3ce-4dee-b442-c1a712027d9e" (UID: "0e44ee83-d3ce-4dee-b442-c1a712027d9e"). InnerVolumeSpecName "kube-api-access-p44t6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:15:00.806165 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:00.806134    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p44t6\" (UniqueName: \"kubernetes.io/projected/0e44ee83-d3ce-4dee-b442-c1a712027d9e-kube-api-access-p44t6\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:15:00.806165 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:00.806160    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e44ee83-d3ce-4dee-b442-c1a712027d9e-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:15:00.806165 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:00.806170    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e44ee83-d3ce-4dee-b442-c1a712027d9e-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:15:00.806418 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:00.806181    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e44ee83-d3ce-4dee-b442-c1a712027d9e-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:15:00.981793 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:15:00.981713    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:15:01.196653 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:01.196614    4942 generic.go:356] "Generic (PLEG): container finished" podID="0e44ee83-d3ce-4dee-b442-c1a712027d9e" containerID="51cd292b0d2c8411c3084877c132d04c5dac7a23ccfe7f757f28ed5f25a937df" exitCode=0
Nov 02 13:15:01.196880 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:01.196675    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2" event={"ID":"0e44ee83-d3ce-4dee-b442-c1a712027d9e","Type":"ContainerDied","Data":"51cd292b0d2c8411c3084877c132d04c5dac7a23ccfe7f757f28ed5f25a937df"}
Nov 02 13:15:01.196880 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:01.196692    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"
Nov 02 13:15:01.196880 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:01.196720    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2" event={"ID":"0e44ee83-d3ce-4dee-b442-c1a712027d9e","Type":"ContainerDied","Data":"f58b649b3fd91b324ce2d6bb033908d79c3b1e861db5f86c437245ce92e6f56b"}
Nov 02 13:15:01.196880 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:01.196742    4942 scope.go:122] "RemoveContainer" containerID="51cd292b0d2c8411c3084877c132d04c5dac7a23ccfe7f757f28ed5f25a937df"
Nov 02 13:15:01.214655 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:01.214637    4942 scope.go:122] "RemoveContainer" containerID="51cd292b0d2c8411c3084877c132d04c5dac7a23ccfe7f757f28ed5f25a937df"
Nov 02 13:15:01.215025 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:15:01.214994    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51cd292b0d2c8411c3084877c132d04c5dac7a23ccfe7f757f28ed5f25a937df\": container with ID starting with 51cd292b0d2c8411c3084877c132d04c5dac7a23ccfe7f757f28ed5f25a937df not found: ID does not exist" containerID="51cd292b0d2c8411c3084877c132d04c5dac7a23ccfe7f757f28ed5f25a937df"
Nov 02 13:15:01.215098 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:01.215042    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51cd292b0d2c8411c3084877c132d04c5dac7a23ccfe7f757f28ed5f25a937df"} err="failed to get container status \"51cd292b0d2c8411c3084877c132d04c5dac7a23ccfe7f757f28ed5f25a937df\": rpc error: code = NotFound desc = could not find container \"51cd292b0d2c8411c3084877c132d04c5dac7a23ccfe7f757f28ed5f25a937df\": container with ID starting with 51cd292b0d2c8411c3084877c132d04c5dac7a23ccfe7f757f28ed5f25a937df not found: ID does not exist"
Nov 02 13:15:01.360415 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:01.360382    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"]
Nov 02 13:15:01.366743 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:01.366703    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c1934578-67c9dd9857-vprj2"]
Nov 02 13:15:01.542680 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:15:01.542650    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e44ee83-d3ce-4dee-b442-c1a712027d9e" path="/var/lib/kubelet/pods/0e44ee83-d3ce-4dee-b442-c1a712027d9e/volumes"
Nov 02 13:15:30.982206 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:15:30.982169    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:16:00.981491 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:16:00.981445    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:16:14.211702 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:14.211594    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:16:14.229109 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:14.229088    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:16:21.974916 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:21.974875    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8"]
Nov 02 13:16:21.975552 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:21.975499    4942 cpu_manager.go:414] "containerMap: removing container" podUID="0e44ee83-d3ce-4dee-b442-c1a712027d9e" containerName="maas-api"
Nov 02 13:16:21.975552 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:21.975517    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e44ee83-d3ce-4dee-b442-c1a712027d9e" containerName="maas-api"
Nov 02 13:16:21.975674 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:21.975612    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e44ee83-d3ce-4dee-b442-c1a712027d9e" containerName="maas-api"
Nov 02 13:16:21.979134 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:21.979113    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8"
Nov 02 13:16:22.019427 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:22.019398    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbbx5\" (UniqueName: \"kubernetes.io/projected/2982eabd-924c-4f43-81bf-bc39e197a79c-kube-api-access-bbbx5\") pod \"maas-api-revoke-sub-f904bff9-zx5t8\" (UID: \"2982eabd-924c-4f43-81bf-bc39e197a79c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8"
Nov 02 13:16:22.019560 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:22.019439    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2982eabd-924c-4f43-81bf-bc39e197a79c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f904bff9-zx5t8\" (UID: \"2982eabd-924c-4f43-81bf-bc39e197a79c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8"
Nov 02 13:16:22.019560 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:22.019520    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2982eabd-924c-4f43-81bf-bc39e197a79c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f904bff9-zx5t8\" (UID: \"2982eabd-924c-4f43-81bf-bc39e197a79c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8"
Nov 02 13:16:22.033220 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:22.032870    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 13:16:22.082221 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:22.082176    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jcthm\""
Nov 02 13:16:22.085875 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:22.085845    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8"]
Nov 02 13:16:22.120340 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:22.120290    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbbx5\" (UniqueName: \"kubernetes.io/projected/2982eabd-924c-4f43-81bf-bc39e197a79c-kube-api-access-bbbx5\") pod \"maas-api-revoke-sub-f904bff9-zx5t8\" (UID: \"2982eabd-924c-4f43-81bf-bc39e197a79c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8"
Nov 02 13:16:22.120459 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:22.120355    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2982eabd-924c-4f43-81bf-bc39e197a79c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f904bff9-zx5t8\" (UID: \"2982eabd-924c-4f43-81bf-bc39e197a79c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8"
Nov 02 13:16:22.120459 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:22.120415    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2982eabd-924c-4f43-81bf-bc39e197a79c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f904bff9-zx5t8\" (UID: \"2982eabd-924c-4f43-81bf-bc39e197a79c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8"
Nov 02 13:16:22.121005 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:22.120984    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2982eabd-924c-4f43-81bf-bc39e197a79c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f904bff9-zx5t8\" (UID: \"2982eabd-924c-4f43-81bf-bc39e197a79c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8"
Nov 02 13:16:22.122754 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:22.122730    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2982eabd-924c-4f43-81bf-bc39e197a79c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f904bff9-zx5t8\" (UID: \"2982eabd-924c-4f43-81bf-bc39e197a79c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8"
Nov 02 13:16:22.181497 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:22.181466    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbbx5\" (UniqueName: \"kubernetes.io/projected/2982eabd-924c-4f43-81bf-bc39e197a79c-kube-api-access-bbbx5\") pod \"maas-api-revoke-sub-f904bff9-zx5t8\" (UID: \"2982eabd-924c-4f43-81bf-bc39e197a79c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8"
Nov 02 13:16:22.292464 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:22.292389    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8"
Nov 02 13:16:22.669160 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:22.669136    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8"]
Nov 02 13:16:22.670875 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:16:22.670840    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2982eabd_924c_4f43_81bf_bc39e197a79c.slice/crio-6632eef893e524e98ca084a50e24df185b8527a731716a890f641bcc4a639c26 WatchSource:0}: Error finding container 6632eef893e524e98ca084a50e24df185b8527a731716a890f641bcc4a639c26: Status 404 returned error can't find the container with id 6632eef893e524e98ca084a50e24df185b8527a731716a890f641bcc4a639c26
Nov 02 13:16:23.636682 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:23.636644    4942 generic.go:356] "Generic (PLEG): container finished" podID="2982eabd-924c-4f43-81bf-bc39e197a79c" containerID="8b0e114ad961d57335f743b66e1b4b745fb339a192faa0cd60b80e44945b1687" exitCode=0
Nov 02 13:16:23.637047 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:23.636688    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8" event={"ID":"2982eabd-924c-4f43-81bf-bc39e197a79c","Type":"ContainerDied","Data":"8b0e114ad961d57335f743b66e1b4b745fb339a192faa0cd60b80e44945b1687"}
Nov 02 13:16:23.637047 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:23.636752    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8" event={"ID":"2982eabd-924c-4f43-81bf-bc39e197a79c","Type":"ContainerStarted","Data":"6632eef893e524e98ca084a50e24df185b8527a731716a890f641bcc4a639c26"}
Nov 02 13:16:24.781521 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:24.781496    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8"
Nov 02 13:16:24.844590 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:24.844555    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2982eabd-924c-4f43-81bf-bc39e197a79c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2982eabd-924c-4f43-81bf-bc39e197a79c-maas-api-openshift-ca\") pod \"2982eabd-924c-4f43-81bf-bc39e197a79c\" (UID: \"2982eabd-924c-4f43-81bf-bc39e197a79c\") "
Nov 02 13:16:24.844722 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:24.844600    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2982eabd-924c-4f43-81bf-bc39e197a79c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2982eabd-924c-4f43-81bf-bc39e197a79c-maas-api-xks-ca\") pod \"2982eabd-924c-4f43-81bf-bc39e197a79c\" (UID: \"2982eabd-924c-4f43-81bf-bc39e197a79c\") "
Nov 02 13:16:24.844722 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:24.844665    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2982eabd-924c-4f43-81bf-bc39e197a79c-kube-api-access-bbbx5\" (UniqueName: \"kubernetes.io/projected/2982eabd-924c-4f43-81bf-bc39e197a79c-kube-api-access-bbbx5\") pod \"2982eabd-924c-4f43-81bf-bc39e197a79c\" (UID: \"2982eabd-924c-4f43-81bf-bc39e197a79c\") "
Nov 02 13:16:24.844946 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:24.844920    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2982eabd-924c-4f43-81bf-bc39e197a79c-maas-api-openshift-ca" pod "2982eabd-924c-4f43-81bf-bc39e197a79c" (UID: "2982eabd-924c-4f43-81bf-bc39e197a79c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:16:24.845056 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:24.845035    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2982eabd-924c-4f43-81bf-bc39e197a79c-maas-api-openshift-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:16:24.846698 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:24.846671    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2982eabd-924c-4f43-81bf-bc39e197a79c-kube-api-access-bbbx5" pod "2982eabd-924c-4f43-81bf-bc39e197a79c" (UID: "2982eabd-924c-4f43-81bf-bc39e197a79c"). InnerVolumeSpecName "kube-api-access-bbbx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:16:24.846804 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:24.846753    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2982eabd-924c-4f43-81bf-bc39e197a79c-maas-api-xks-ca" pod "2982eabd-924c-4f43-81bf-bc39e197a79c" (UID: "2982eabd-924c-4f43-81bf-bc39e197a79c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:16:24.945660 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:24.945595    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2982eabd-924c-4f43-81bf-bc39e197a79c-maas-api-xks-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:16:24.945660 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:24.945622    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbbx5\" (UniqueName: \"kubernetes.io/projected/2982eabd-924c-4f43-81bf-bc39e197a79c-kube-api-access-bbbx5\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:16:25.648357 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:25.648293    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8" event={"ID":"2982eabd-924c-4f43-81bf-bc39e197a79c","Type":"ContainerDied","Data":"6632eef893e524e98ca084a50e24df185b8527a731716a890f641bcc4a639c26"}
Nov 02 13:16:25.648357 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:25.648349    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8"
Nov 02 13:16:25.648544 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:25.648372    4942 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6632eef893e524e98ca084a50e24df185b8527a731716a890f641bcc4a639c26"
Nov 02 13:16:26.594505 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.594466    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"]
Nov 02 13:16:26.595080 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.595049    4942 cpu_manager.go:414] "containerMap: removing container" podUID="2982eabd-924c-4f43-81bf-bc39e197a79c" containerName="revoke-keys"
Nov 02 13:16:26.595080 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.595078    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2982eabd-924c-4f43-81bf-bc39e197a79c" containerName="revoke-keys"
Nov 02 13:16:26.595265 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.595210    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="2982eabd-924c-4f43-81bf-bc39e197a79c" containerName="revoke-keys"
Nov 02 13:16:26.598093 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.598072    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.644391 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.644364    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"]
Nov 02 13:16:26.647804 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.647786    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.662036 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.662007    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.662165 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.662054    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.662165 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.662126    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.662273 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.662176    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.662273 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.662198    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.662388 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.662275    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.662388 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.662367    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.662488 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.662399    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jcvs\" (UniqueName: \"kubernetes.io/projected/cc69931c-3e6a-4a50-b706-5805a16167b0-kube-api-access-9jcvs\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.662488 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.662438    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.662579 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.662507    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eee44d9a-da91-4564-8b1e-efda1f126db6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.662579 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.662527    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.662579 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.662550    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc69931c-3e6a-4a50-b706-5805a16167b0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.662719 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.662603    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwjcb\" (UniqueName: \"kubernetes.io/projected/eee44d9a-da91-4564-8b1e-efda1f126db6-kube-api-access-gwjcb\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.662719 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.662655    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.662719 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.662680    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.662719 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.662700    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.662903 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.662720    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.662903 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.662759    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.704036 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.704002    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"]
Nov 02 13:16:26.755216 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.755179    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"]
Nov 02 13:16:26.764019 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.763986    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.764133 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764032    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.764133 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764062    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.764133 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764092    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.764286 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764133    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jcvs\" (UniqueName: \"kubernetes.io/projected/cc69931c-3e6a-4a50-b706-5805a16167b0-kube-api-access-9jcvs\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.764286 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764193    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.764286 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764227    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eee44d9a-da91-4564-8b1e-efda1f126db6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.764286 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764254    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.764502 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764288    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc69931c-3e6a-4a50-b706-5805a16167b0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.764502 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764351    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwjcb\" (UniqueName: \"kubernetes.io/projected/eee44d9a-da91-4564-8b1e-efda1f126db6-kube-api-access-gwjcb\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.764502 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764396    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.764502 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764437    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.764502 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764468    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.764502 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764497    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.765220 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764529    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.765220 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764537    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.765220 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764561    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.765220 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764603    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.765220 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764626    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.765220 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.764849    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.765220 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.765016    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.765615 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.765275    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.765615 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.765441    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.765615 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.765469    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc69931c-3e6a-4a50-b706-5805a16167b0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.765782 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.765712    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.765782 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.765770    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.765960 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.765937    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eee44d9a-da91-4564-8b1e-efda1f126db6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.766075 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.766053    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.767101 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.767075    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.767209 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.767181    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.767475 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.767452    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.767748 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.767729    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.828661 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.828595    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.828661 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.828612    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.829035 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.829005    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwjcb\" (UniqueName: \"kubernetes.io/projected/eee44d9a-da91-4564-8b1e-efda1f126db6-kube-api-access-gwjcb\") pod \"maas-default-gateway-openshift-default-6dcf757899-8gqnt\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:26.829501 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.829420    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jcvs\" (UniqueName: \"kubernetes.io/projected/cc69931c-3e6a-4a50-b706-5805a16167b0-kube-api-access-9jcvs\") pod \"maas-default-gateway-openshift-default-6dcf757899-r8lg8\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.911399 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.911369    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:26.963572 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:26.963522    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:27.338161 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:27.338136    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"]
Nov 02 13:16:27.339046 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:16:27.339016    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc69931c_3e6a_4a50_b706_5805a16167b0.slice/crio-f781fbafa5b34be8b22c0b16f099549fa92e58e2e80204917cdd89c912000674 WatchSource:0}: Error finding container f781fbafa5b34be8b22c0b16f099549fa92e58e2e80204917cdd89c912000674: Status 404 returned error can't find the container with id f781fbafa5b34be8b22c0b16f099549fa92e58e2e80204917cdd89c912000674
Nov 02 13:16:27.341149 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:27.341112    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:16:27.341239 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:27.341193    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:16:27.341239 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:27.341225    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:16:27.389664 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:27.389641    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"]
Nov 02 13:16:27.397268 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:16:27.397245    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee44d9a_da91_4564_8b1e_efda1f126db6.slice/crio-49f3bd7aa0189e991142575bd0575f74285dd9206f9ee7dd14ab0d5061eca7ea WatchSource:0}: Error finding container 49f3bd7aa0189e991142575bd0575f74285dd9206f9ee7dd14ab0d5061eca7ea: Status 404 returned error can't find the container with id 49f3bd7aa0189e991142575bd0575f74285dd9206f9ee7dd14ab0d5061eca7ea
Nov 02 13:16:27.399583 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:27.399552    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:16:27.399671 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:27.399615    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:16:27.399671 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:27.399644    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:16:27.663277 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:27.663234    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt" event={"ID":"eee44d9a-da91-4564-8b1e-efda1f126db6","Type":"ContainerStarted","Data":"4392ce405ba65678f2a09c51bf86304d0ed48033e3775d65efbb4bb9e76c90e6"}
Nov 02 13:16:27.663740 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:27.663321    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt" event={"ID":"eee44d9a-da91-4564-8b1e-efda1f126db6","Type":"ContainerStarted","Data":"49f3bd7aa0189e991142575bd0575f74285dd9206f9ee7dd14ab0d5061eca7ea"}
Nov 02 13:16:27.665225 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:27.665192    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8" event={"ID":"cc69931c-3e6a-4a50-b706-5805a16167b0","Type":"ContainerStarted","Data":"4940097a7fac4f1e49271ba6f31935a1f3b8998c31ca7a374c1fbfdfee8d4fe6"}
Nov 02 13:16:27.665389 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:27.665245    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8" event={"ID":"cc69931c-3e6a-4a50-b706-5805a16167b0","Type":"ContainerStarted","Data":"f781fbafa5b34be8b22c0b16f099549fa92e58e2e80204917cdd89c912000674"}
Nov 02 13:16:27.785344 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:27.785263    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt" podStartSLOduration=1.78525014 podStartE2EDuration="1.78525014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:16:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:16:27.782735898 +0000 UTC m=+4814.354085657" watchObservedRunningTime="2026-11-02 13:16:27.78525014 +0000 UTC m=+4814.356599896"
Nov 02 13:16:27.904470 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:27.904414    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8" podStartSLOduration=1.904396884 podStartE2EDuration="1.904396884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:16:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:16:27.901970643 +0000 UTC m=+4814.473320402" watchObservedRunningTime="2026-11-02 13:16:27.904396884 +0000 UTC m=+4814.475746641"
Nov 02 13:16:27.912348 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:27.912284    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:27.967181 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:27.967077    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:28.913578 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:28.913541    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-r8lg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 02 13:16:28.913985 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:28.913596    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8" podUID="cc69931c-3e6a-4a50-b706-5805a16167b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 02 13:16:28.971322 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:28.971087    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-8gqnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body=
Nov 02 13:16:28.971322 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:28.971162    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt" podUID="eee44d9a-da91-4564-8b1e-efda1f126db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused"
Nov 02 13:16:29.912033 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:29.911978    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-r8lg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 02 13:16:29.912399 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:29.912374    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8" podUID="cc69931c-3e6a-4a50-b706-5805a16167b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 02 13:16:29.968323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:29.965667    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-8gqnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body=
Nov 02 13:16:29.968323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:29.965729    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt" podUID="eee44d9a-da91-4564-8b1e-efda1f126db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused"
Nov 02 13:16:30.911976 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:30.911931    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-r8lg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 02 13:16:30.912187 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:30.911991    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8" podUID="cc69931c-3e6a-4a50-b706-5805a16167b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 02 13:16:30.965844 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:30.965809    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-8gqnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body=
Nov 02 13:16:30.966153 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:30.966104    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt" podUID="eee44d9a-da91-4564-8b1e-efda1f126db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused"
Nov 02 13:16:31.030810 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:16:31.030771    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:16:31.912337 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:31.911923    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-r8lg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 02 13:16:31.912337 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:31.911984    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8" podUID="cc69931c-3e6a-4a50-b706-5805a16167b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 02 13:16:31.992798 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:31.992666    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-8gqnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body=
Nov 02 13:16:31.992798 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:31.992729    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt" podUID="eee44d9a-da91-4564-8b1e-efda1f126db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused"
Nov 02 13:16:32.912204 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:32.912142    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-r8lg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 02 13:16:32.912851 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:32.912808    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8" podUID="cc69931c-3e6a-4a50-b706-5805a16167b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 02 13:16:32.969740 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:32.969703    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-8gqnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body=
Nov 02 13:16:32.970066 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:32.970025    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt" podUID="eee44d9a-da91-4564-8b1e-efda1f126db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused"
Nov 02 13:16:33.912219 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:33.912175    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-r8lg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 02 13:16:33.912751 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:33.912263    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8" podUID="cc69931c-3e6a-4a50-b706-5805a16167b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 02 13:16:33.965079 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:33.965039    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-8gqnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body=
Nov 02 13:16:33.965377 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:33.965338    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt" podUID="eee44d9a-da91-4564-8b1e-efda1f126db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused"
Nov 02 13:16:34.911933 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:34.911884    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-r8lg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Nov 02 13:16:34.912098 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:34.911960    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8" podUID="cc69931c-3e6a-4a50-b706-5805a16167b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Nov 02 13:16:34.965278 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:34.965238    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-8gqnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body=
Nov 02 13:16:34.965689 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:34.965325    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt" podUID="eee44d9a-da91-4564-8b1e-efda1f126db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused"
Nov 02 13:16:36.911737 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:36.911682    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-r8lg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:16:36.912229 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:36.912128    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8" podUID="cc69931c-3e6a-4a50-b706-5805a16167b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:16:36.967501 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:36.967461    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-8gqnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:16:36.967793 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:36.967761    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt" podUID="eee44d9a-da91-4564-8b1e-efda1f126db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:16:37.913171 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:37.913135    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-r8lg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:16:37.913690 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:37.913194    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8" podUID="cc69931c-3e6a-4a50-b706-5805a16167b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:16:37.969053 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:37.969001    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-8gqnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:16:37.969253 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:37.969090    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt" podUID="eee44d9a-da91-4564-8b1e-efda1f126db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:16:38.959322 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:38.957441    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-r8lg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:16:38.959322 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:38.957518    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8" podUID="cc69931c-3e6a-4a50-b706-5805a16167b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:16:39.011579 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:39.011502    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-8gqnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:16:39.011959 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:39.011893    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt" podUID="eee44d9a-da91-4564-8b1e-efda1f126db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:16:39.998478 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:39.998426    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-r8lg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:16:39.998986 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:39.998497    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8" podUID="cc69931c-3e6a-4a50-b706-5805a16167b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:16:40.054496 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:40.054446    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-8gqnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:16:40.054788 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:40.054742    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt" podUID="eee44d9a-da91-4564-8b1e-efda1f126db6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:16:40.059446 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:40.059423    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:40.059839 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:40.059815    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:40.060481 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:40.060459    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:16:40.466360 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:40.466331    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:40.466746 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:40.466716    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:40.468436 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:40.468417    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:16:56.852481 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.852449    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"]
Nov 02 13:16:56.855871 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.855853    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.868824 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.868800    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.868947 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.868872    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7tlf\" (UniqueName: \"kubernetes.io/projected/e8b30edd-8440-4c10-9c73-678c72c86222-kube-api-access-w7tlf\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.868947 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.868905    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8b30edd-8440-4c10-9c73-678c72c86222-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.868947 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.868925    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.869057 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.868951    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.869057 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.869029    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8b30edd-8440-4c10-9c73-678c72c86222-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.869120 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.869077    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8b30edd-8440-4c10-9c73-678c72c86222-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.869157 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.869127    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.869193 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.869159    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.966423 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.966383    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"]
Nov 02 13:16:56.972126 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.971928    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7tlf\" (UniqueName: \"kubernetes.io/projected/e8b30edd-8440-4c10-9c73-678c72c86222-kube-api-access-w7tlf\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.972269 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.972160    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8b30edd-8440-4c10-9c73-678c72c86222-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.972269 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.972188    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.972269 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.972209    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.972269 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.972242    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8b30edd-8440-4c10-9c73-678c72c86222-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.973476 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.973448    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.973619 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.973490    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8b30edd-8440-4c10-9c73-678c72c86222-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.973619 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.973530    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.973619 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.973569    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.973813 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.973622    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.974050 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.974028    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.974247 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.974225    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.974345 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.974243    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8b30edd-8440-4c10-9c73-678c72c86222-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.974535 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.974503    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.975175 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.975153    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:56.976097 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:56.976074    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8b30edd-8440-4c10-9c73-678c72c86222-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:57.033033 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:57.033006    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7tlf\" (UniqueName: \"kubernetes.io/projected/e8b30edd-8440-4c10-9c73-678c72c86222-kube-api-access-w7tlf\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:57.034346 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:57.034325    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8b30edd-8440-4c10-9c73-678c72c86222-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-wkxz9\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:57.168936 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:57.168861    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:57.595934 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:57.595911    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"]
Nov 02 13:16:57.596938 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:16:57.596914    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8b30edd_8440_4c10_9c73_678c72c86222.slice/crio-9086aa4458ef5b09622cb16e7ae5ffb978300187bc976d2b56fa4b4ec3400cd4 WatchSource:0}: Error finding container 9086aa4458ef5b09622cb16e7ae5ffb978300187bc976d2b56fa4b4ec3400cd4: Status 404 returned error can't find the container with id 9086aa4458ef5b09622cb16e7ae5ffb978300187bc976d2b56fa4b4ec3400cd4
Nov 02 13:16:57.598921 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:57.598890    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:16:57.598999 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:57.598958    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:16:57.598999 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:57.598989    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:16:57.943706 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:57.943667    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" event={"ID":"e8b30edd-8440-4c10-9c73-678c72c86222","Type":"ContainerStarted","Data":"7c7ca367d19d30279ac69503d1406d6f31429a22e4577d425f67f71ac45f4cba"}
Nov 02 13:16:57.944094 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:57.943729    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" event={"ID":"e8b30edd-8440-4c10-9c73-678c72c86222","Type":"ContainerStarted","Data":"9086aa4458ef5b09622cb16e7ae5ffb978300187bc976d2b56fa4b4ec3400cd4"}
Nov 02 13:16:58.170081 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:58.170044    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:16:58.173158 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:58.173124    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wkxz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body=
Nov 02 13:16:58.173450 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:58.173421    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused"
Nov 02 13:16:59.172323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:59.171652    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wkxz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body=
Nov 02 13:16:59.172323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:16:59.171720    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused"
Nov 02 13:17:00.172693 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:00.172651    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wkxz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body=
Nov 02 13:17:00.173334 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:00.173273    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused"
Nov 02 13:17:01.003413 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:17:01.003381    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:17:01.174075 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:01.173641    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wkxz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body=
Nov 02 13:17:01.174075 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:01.173703    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused"
Nov 02 13:17:02.173332 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:02.172631    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wkxz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body=
Nov 02 13:17:02.173332 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:02.172712    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused"
Nov 02 13:17:03.177324 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:03.174631    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wkxz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body=
Nov 02 13:17:03.177324 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:03.174693    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused"
Nov 02 13:17:04.169572 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:04.169524    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wkxz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body=
Nov 02 13:17:04.169742 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:04.169600    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused"
Nov 02 13:17:05.169426 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:05.169389    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wkxz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body=
Nov 02 13:17:05.169830 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:05.169445    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused"
Nov 02 13:17:07.170891 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:07.170854    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wkxz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:17:07.171345 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:07.170917    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:17:08.171276 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:08.171234    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wkxz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:17:08.171869 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:08.171318    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:17:09.212479 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:09.212431    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wkxz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:17:09.213014 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:09.212495    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:17:09.827340 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:09.827274    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:17:09.827708 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:09.827684    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:17:09.828472 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:09.828450    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:17:10.020920 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:10.020873    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" podStartSLOduration=14.020860178 podStartE2EDuration="14.020860178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:16:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:16:58.068588759 +0000 UTC m=+4844.639938520" watchObservedRunningTime="2026-11-02 13:17:10.020860178 +0000 UTC m=+4856.592209935"
Nov 02 13:17:17.690323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:17.689391    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-7hsh2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:17:17.690323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:17:17.689529    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:17:30.982389 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:17:30.982351    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:18:00.983256 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:18:00.983176    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:18:30.996930 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:18:30.996879    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:18:32.689322 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:32.688391    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-7hsh2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:18:32.689322 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:32.688455    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:18:42.282413 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:42.282374    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs"]
Nov 02 13:18:42.286117 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:42.286100    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs"
Nov 02 13:18:42.339342 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:42.339290    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 13:18:42.342940 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:42.342914    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e49cfee7-pvtcs\" (UID: \"e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs"
Nov 02 13:18:42.343098 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:42.342960    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmnwk\" (UniqueName: \"kubernetes.io/projected/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-kube-api-access-dmnwk\") pod \"maas-api-revoke-sub-e49cfee7-pvtcs\" (UID: \"e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs"
Nov 02 13:18:42.343098 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:42.343006    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e49cfee7-pvtcs\" (UID: \"e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs"
Nov 02 13:18:42.388367 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:42.388338    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jcthm\""
Nov 02 13:18:42.390892 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:42.390876    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs"]
Nov 02 13:18:42.443629 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:42.443606    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e49cfee7-pvtcs\" (UID: \"e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs"
Nov 02 13:18:42.443755 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:42.443642    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmnwk\" (UniqueName: \"kubernetes.io/projected/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-kube-api-access-dmnwk\") pod \"maas-api-revoke-sub-e49cfee7-pvtcs\" (UID: \"e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs"
Nov 02 13:18:42.443796 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:42.443758    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e49cfee7-pvtcs\" (UID: \"e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs"
Nov 02 13:18:42.444414 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:42.444396    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e49cfee7-pvtcs\" (UID: \"e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs"
Nov 02 13:18:42.445853 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:42.445836    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e49cfee7-pvtcs\" (UID: \"e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs"
Nov 02 13:18:42.506909 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:42.506884    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmnwk\" (UniqueName: \"kubernetes.io/projected/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-kube-api-access-dmnwk\") pod \"maas-api-revoke-sub-e49cfee7-pvtcs\" (UID: \"e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs"
Nov 02 13:18:42.598952 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:42.598892    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs"
Nov 02 13:18:42.976180 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:42.976158    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs"]
Nov 02 13:18:42.978488 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:18:42.978460    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3d787c6_07a7_4b8f_8ca6_7eeb7ab66e48.slice/crio-a80ca474031d4bfded4459ddcad0f64570071e97782dc73838d929ed9a48082b WatchSource:0}: Error finding container a80ca474031d4bfded4459ddcad0f64570071e97782dc73838d929ed9a48082b: Status 404 returned error can't find the container with id a80ca474031d4bfded4459ddcad0f64570071e97782dc73838d929ed9a48082b
Nov 02 13:18:43.877969 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:43.877932    4942 generic.go:356] "Generic (PLEG): container finished" podID="e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48" containerID="4d432c054c7a55aa8e4f7a57a84235e9c970acfb9757838758d25fe70542f05d" exitCode=0
Nov 02 13:18:43.878361 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:43.878024    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs" event={"ID":"e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48","Type":"ContainerDied","Data":"4d432c054c7a55aa8e4f7a57a84235e9c970acfb9757838758d25fe70542f05d"}
Nov 02 13:18:43.878361 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:43.878085    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs" event={"ID":"e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48","Type":"ContainerStarted","Data":"a80ca474031d4bfded4459ddcad0f64570071e97782dc73838d929ed9a48082b"}
Nov 02 13:18:45.025578 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:45.025553    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs"
Nov 02 13:18:45.066540 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:45.066515    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-maas-api-xks-ca\") pod \"e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48\" (UID: \"e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48\") "
Nov 02 13:18:45.066654 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:45.066582    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-kube-api-access-dmnwk\" (UniqueName: \"kubernetes.io/projected/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-kube-api-access-dmnwk\") pod \"e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48\" (UID: \"e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48\") "
Nov 02 13:18:45.066705 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:45.066667    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-maas-api-openshift-ca\") pod \"e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48\" (UID: \"e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48\") "
Nov 02 13:18:45.067033 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:45.066996    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-maas-api-openshift-ca" pod "e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48" (UID: "e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:18:45.068484 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:45.068457    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-maas-api-xks-ca" pod "e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48" (UID: "e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:18:45.068626 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:45.068605    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-kube-api-access-dmnwk" pod "e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48" (UID: "e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48"). InnerVolumeSpecName "kube-api-access-dmnwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:18:45.168780 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:45.168701    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-maas-api-xks-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:18:45.168780 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:45.168742    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmnwk\" (UniqueName: \"kubernetes.io/projected/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-kube-api-access-dmnwk\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:18:45.168780 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:45.168759    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48-maas-api-openshift-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:18:45.890236 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:45.890211    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs"
Nov 02 13:18:45.890429 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:45.890253    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs" event={"ID":"e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48","Type":"ContainerDied","Data":"a80ca474031d4bfded4459ddcad0f64570071e97782dc73838d929ed9a48082b"}
Nov 02 13:18:45.890429 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:18:45.890323    4942 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a80ca474031d4bfded4459ddcad0f64570071e97782dc73838d929ed9a48082b"
Nov 02 13:19:00.923160 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:00.923116    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9"]
Nov 02 13:19:00.924898 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:00.924869    4942 cpu_manager.go:414] "containerMap: removing container" podUID="e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48" containerName="revoke-keys"
Nov 02 13:19:00.925067 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:00.925050    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48" containerName="revoke-keys"
Nov 02 13:19:00.925489 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:00.925471    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48" containerName="revoke-keys"
Nov 02 13:19:00.930481 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:00.930453    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9"
Nov 02 13:19:00.972994 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:00.972956    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9"]
Nov 02 13:19:01.020883 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:01.020847    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jcthm\""
Nov 02 13:19:01.021285 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:01.021265    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 02 13:19:01.023169 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:01.023143    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5xj9\" (UniqueName: \"kubernetes.io/projected/c8726a12-ed2b-440d-9525-db2a39d769ec-kube-api-access-x5xj9\") pod \"maas-api-revoke-sub-7d5a4698-gc9w9\" (UID: \"c8726a12-ed2b-440d-9525-db2a39d769ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9"
Nov 02 13:19:01.023468 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:01.023436    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c8726a12-ed2b-440d-9525-db2a39d769ec-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7d5a4698-gc9w9\" (UID: \"c8726a12-ed2b-440d-9525-db2a39d769ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9"
Nov 02 13:19:01.023845 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:01.023807    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c8726a12-ed2b-440d-9525-db2a39d769ec-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7d5a4698-gc9w9\" (UID: \"c8726a12-ed2b-440d-9525-db2a39d769ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9"
Nov 02 13:19:01.025032 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:19:01.025010    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:19:01.125216 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:01.125150    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5xj9\" (UniqueName: \"kubernetes.io/projected/c8726a12-ed2b-440d-9525-db2a39d769ec-kube-api-access-x5xj9\") pod \"maas-api-revoke-sub-7d5a4698-gc9w9\" (UID: \"c8726a12-ed2b-440d-9525-db2a39d769ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9"
Nov 02 13:19:01.125216 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:01.125204    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c8726a12-ed2b-440d-9525-db2a39d769ec-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7d5a4698-gc9w9\" (UID: \"c8726a12-ed2b-440d-9525-db2a39d769ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9"
Nov 02 13:19:01.125513 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:01.125353    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c8726a12-ed2b-440d-9525-db2a39d769ec-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7d5a4698-gc9w9\" (UID: \"c8726a12-ed2b-440d-9525-db2a39d769ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9"
Nov 02 13:19:01.126290 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:01.126242    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c8726a12-ed2b-440d-9525-db2a39d769ec-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7d5a4698-gc9w9\" (UID: \"c8726a12-ed2b-440d-9525-db2a39d769ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9"
Nov 02 13:19:01.143449 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:01.143379    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c8726a12-ed2b-440d-9525-db2a39d769ec-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7d5a4698-gc9w9\" (UID: \"c8726a12-ed2b-440d-9525-db2a39d769ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9"
Nov 02 13:19:01.202155 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:01.202047    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5xj9\" (UniqueName: \"kubernetes.io/projected/c8726a12-ed2b-440d-9525-db2a39d769ec-kube-api-access-x5xj9\") pod \"maas-api-revoke-sub-7d5a4698-gc9w9\" (UID: \"c8726a12-ed2b-440d-9525-db2a39d769ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9"
Nov 02 13:19:01.254757 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:01.254474    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9"
Nov 02 13:19:01.669141 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:01.669101    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9"]
Nov 02 13:19:01.672382 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:19:01.672351    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8726a12_ed2b_440d_9525_db2a39d769ec.slice/crio-0e51783259fa58f81b2f904dcb719e5d6c937b34081a3b3a752374f1c2422c70 WatchSource:0}: Error finding container 0e51783259fa58f81b2f904dcb719e5d6c937b34081a3b3a752374f1c2422c70: Status 404 returned error can't find the container with id 0e51783259fa58f81b2f904dcb719e5d6c937b34081a3b3a752374f1c2422c70
Nov 02 13:19:02.014321 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:02.013376    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9" event={"ID":"c8726a12-ed2b-440d-9525-db2a39d769ec","Type":"ContainerStarted","Data":"0e51783259fa58f81b2f904dcb719e5d6c937b34081a3b3a752374f1c2422c70"}
Nov 02 13:19:03.037323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:03.036497    4942 generic.go:356] "Generic (PLEG): container finished" podID="c8726a12-ed2b-440d-9525-db2a39d769ec" containerID="316df60c3aef8a9672661038abf8a852f224e9047df466a41ff41a8bf3823d90" exitCode=0
Nov 02 13:19:03.037323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:03.036576    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9" event={"ID":"c8726a12-ed2b-440d-9525-db2a39d769ec","Type":"ContainerDied","Data":"316df60c3aef8a9672661038abf8a852f224e9047df466a41ff41a8bf3823d90"}
Nov 02 13:19:04.607820 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:04.607787    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9"
Nov 02 13:19:04.774846 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:04.774811    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8726a12-ed2b-440d-9525-db2a39d769ec-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c8726a12-ed2b-440d-9525-db2a39d769ec-maas-api-openshift-ca\") pod \"c8726a12-ed2b-440d-9525-db2a39d769ec\" (UID: \"c8726a12-ed2b-440d-9525-db2a39d769ec\") "
Nov 02 13:19:04.775039 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:04.774903    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c8726a12-ed2b-440d-9525-db2a39d769ec-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c8726a12-ed2b-440d-9525-db2a39d769ec-maas-api-xks-ca\") pod \"c8726a12-ed2b-440d-9525-db2a39d769ec\" (UID: \"c8726a12-ed2b-440d-9525-db2a39d769ec\") "
Nov 02 13:19:04.775039 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:04.774932    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8726a12-ed2b-440d-9525-db2a39d769ec-kube-api-access-x5xj9\" (UniqueName: \"kubernetes.io/projected/c8726a12-ed2b-440d-9525-db2a39d769ec-kube-api-access-x5xj9\") pod \"c8726a12-ed2b-440d-9525-db2a39d769ec\" (UID: \"c8726a12-ed2b-440d-9525-db2a39d769ec\") "
Nov 02 13:19:04.775284 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:04.775251    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8726a12-ed2b-440d-9525-db2a39d769ec-maas-api-openshift-ca" pod "c8726a12-ed2b-440d-9525-db2a39d769ec" (UID: "c8726a12-ed2b-440d-9525-db2a39d769ec"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:19:04.777065 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:04.777041    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8726a12-ed2b-440d-9525-db2a39d769ec-maas-api-xks-ca" pod "c8726a12-ed2b-440d-9525-db2a39d769ec" (UID: "c8726a12-ed2b-440d-9525-db2a39d769ec"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:19:04.777190 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:04.777106    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8726a12-ed2b-440d-9525-db2a39d769ec-kube-api-access-x5xj9" pod "c8726a12-ed2b-440d-9525-db2a39d769ec" (UID: "c8726a12-ed2b-440d-9525-db2a39d769ec"). InnerVolumeSpecName "kube-api-access-x5xj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:19:04.876022 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:04.875993    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c8726a12-ed2b-440d-9525-db2a39d769ec-maas-api-openshift-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:19:04.876022 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:04.876022    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c8726a12-ed2b-440d-9525-db2a39d769ec-maas-api-xks-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:19:04.876200 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:04.876033    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5xj9\" (UniqueName: \"kubernetes.io/projected/c8726a12-ed2b-440d-9525-db2a39d769ec-kube-api-access-x5xj9\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:19:05.060436 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:05.060395    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9" event={"ID":"c8726a12-ed2b-440d-9525-db2a39d769ec","Type":"ContainerDied","Data":"0e51783259fa58f81b2f904dcb719e5d6c937b34081a3b3a752374f1c2422c70"}
Nov 02 13:19:05.060580 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:05.060463    4942 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e51783259fa58f81b2f904dcb719e5d6c937b34081a3b3a752374f1c2422c70"
Nov 02 13:19:05.060580 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:05.060414    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9"
Nov 02 13:19:14.037388 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.037355    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"]
Nov 02 13:19:14.037791 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.037782    4942 cpu_manager.go:414] "containerMap: removing container" podUID="c8726a12-ed2b-440d-9525-db2a39d769ec" containerName="revoke-keys"
Nov 02 13:19:14.037837 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.037794    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8726a12-ed2b-440d-9525-db2a39d769ec" containerName="revoke-keys"
Nov 02 13:19:14.037869 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.037855    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8726a12-ed2b-440d-9525-db2a39d769ec" containerName="revoke-keys"
Nov 02 13:19:14.041266 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.041238    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.060779 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.060740    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0ffdc12-01f9-420e-95de-d980a47b4334-istiod-ca-cert\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.061049 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.061009    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-workload-socket\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.061189 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.061072    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-credential-socket\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.061189 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.061104    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-data\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.061189 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.061132    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-podinfo\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.061189 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.061191    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-workload-certs\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.061498 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.061229    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glsrv\" (UniqueName: \"kubernetes.io/projected/d0ffdc12-01f9-420e-95de-d980a47b4334-kube-api-access-glsrv\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.061498 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.061267    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-token\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.061498 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.061352    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-envoy\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.095152 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.095118    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-62ca10-openshift-default-dockercfg-qbg7p\""
Nov 02 13:19:14.161781 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.161750    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-workload-certs\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.161954 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.161792    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glsrv\" (UniqueName: \"kubernetes.io/projected/d0ffdc12-01f9-420e-95de-d980a47b4334-kube-api-access-glsrv\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.161954 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.161816    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-token\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.161954 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.161860    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-envoy\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.161954 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.161895    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0ffdc12-01f9-420e-95de-d980a47b4334-istiod-ca-cert\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.161954 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.161926    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-workload-socket\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.162177 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.161968    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-credential-socket\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.162177 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.161997    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-data\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.162177 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.162119    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-podinfo\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.162358 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.162339    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-workload-certs\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.162415 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.162382    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-data\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.162544 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.162523    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-workload-socket\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.162610 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.162554    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-credential-socket\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.162739 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.162721    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0ffdc12-01f9-420e-95de-d980a47b4334-istiod-ca-cert\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.164281 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.164254    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-envoy\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.164553 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.164532    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-podinfo\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.187280 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.187248    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"]
Nov 02 13:19:14.223214 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.223175    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-token\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.231672 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.231645    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glsrv\" (UniqueName: \"kubernetes.io/projected/d0ffdc12-01f9-420e-95de-d980a47b4334-kube-api-access-glsrv\") pod \"e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") " pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.360579 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.360549    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:14.805817 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.805792    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"]
Nov 02 13:19:14.807440 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:19:14.807409    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ffdc12_01f9_420e_95de_d980a47b4334.slice/crio-42006b03fad9d08003a944bf5d2ec2f37ead55a93157277f948748f265b3ea48 WatchSource:0}: Error finding container 42006b03fad9d08003a944bf5d2ec2f37ead55a93157277f948748f265b3ea48: Status 404 returned error can't find the container with id 42006b03fad9d08003a944bf5d2ec2f37ead55a93157277f948748f265b3ea48
Nov 02 13:19:14.809571 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.809538    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:19:14.809659 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.809608    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:19:14.809659 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:14.809639    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:19:15.117137 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:15.117104    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6" event={"ID":"d0ffdc12-01f9-420e-95de-d980a47b4334","Type":"ContainerStarted","Data":"b1fcc0a591bcb67169c550b1e49bf64d06431d278a36806a11eddea2a7fdcdc6"}
Nov 02 13:19:15.117540 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:15.117155    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6" event={"ID":"d0ffdc12-01f9-420e-95de-d980a47b4334","Type":"ContainerStarted","Data":"42006b03fad9d08003a944bf5d2ec2f37ead55a93157277f948748f265b3ea48"}
Nov 02 13:19:15.239959 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:15.239882    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6" podStartSLOduration=2.239864529 podStartE2EDuration="2.239864529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:19:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:19:15.237511262 +0000 UTC m=+4981.808861019" watchObservedRunningTime="2026-11-02 13:19:15.239864529 +0000 UTC m=+4981.811214283"
Nov 02 13:19:15.362335 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:15.362199    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:15.369846 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:15.369747    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:16.122913 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:16.122886    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:16.123745 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:16.123725    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:22.121382 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.121349    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np"]
Nov 02 13:19:22.125186 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.125163    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np"
Nov 02 13:19:22.178312 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.178271    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-62ca10\""
Nov 02 13:19:22.230398 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.230366    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np"]
Nov 02 13:19:22.283885 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.283848    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk"]
Nov 02 13:19:22.293230 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.293198    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk"
Nov 02 13:19:22.297865 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.297835    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9v4k\" (UniqueName: \"kubernetes.io/projected/7d136b78-8365-465e-851c-962275c7b35e-kube-api-access-h9v4k\") pod \"payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np\" (UID: \"7d136b78-8365-465e-851c-962275c7b35e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np"
Nov 02 13:19:22.298016 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.297912    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d136b78-8365-465e-851c-962275c7b35e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np\" (UID: \"7d136b78-8365-465e-851c-962275c7b35e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np"
Nov 02 13:19:22.341942 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.341909    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk"]
Nov 02 13:19:22.351832 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.351804    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-6939e544-dockercfg-zxqtb\""
Nov 02 13:19:22.399549 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.399457    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9v4k\" (UniqueName: \"kubernetes.io/projected/7d136b78-8365-465e-851c-962275c7b35e-kube-api-access-h9v4k\") pod \"payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np\" (UID: \"7d136b78-8365-465e-851c-962275c7b35e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np"
Nov 02 13:19:22.399723 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.399544    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d136b78-8365-465e-851c-962275c7b35e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np\" (UID: \"7d136b78-8365-465e-851c-962275c7b35e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np"
Nov 02 13:19:22.399723 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.399619    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51a7e3aa-25a3-42ee-bed6-818c57159227-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk\" (UID: \"51a7e3aa-25a3-42ee-bed6-818c57159227\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk"
Nov 02 13:19:22.399723 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.399660    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf9vp\" (UniqueName: \"kubernetes.io/projected/51a7e3aa-25a3-42ee-bed6-818c57159227-kube-api-access-cf9vp\") pod \"payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk\" (UID: \"51a7e3aa-25a3-42ee-bed6-818c57159227\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk"
Nov 02 13:19:22.400331 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.400271    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d136b78-8365-465e-851c-962275c7b35e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np\" (UID: \"7d136b78-8365-465e-851c-962275c7b35e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np"
Nov 02 13:19:22.461730 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.461691    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9v4k\" (UniqueName: \"kubernetes.io/projected/7d136b78-8365-465e-851c-962275c7b35e-kube-api-access-h9v4k\") pod \"payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np\" (UID: \"7d136b78-8365-465e-851c-962275c7b35e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np"
Nov 02 13:19:22.500320 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.500269    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51a7e3aa-25a3-42ee-bed6-818c57159227-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk\" (UID: \"51a7e3aa-25a3-42ee-bed6-818c57159227\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk"
Nov 02 13:19:22.500320 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.500323    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf9vp\" (UniqueName: \"kubernetes.io/projected/51a7e3aa-25a3-42ee-bed6-818c57159227-kube-api-access-cf9vp\") pod \"payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk\" (UID: \"51a7e3aa-25a3-42ee-bed6-818c57159227\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk"
Nov 02 13:19:22.500965 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.500944    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51a7e3aa-25a3-42ee-bed6-818c57159227-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk\" (UID: \"51a7e3aa-25a3-42ee-bed6-818c57159227\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk"
Nov 02 13:19:22.562484 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.562450    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf9vp\" (UniqueName: \"kubernetes.io/projected/51a7e3aa-25a3-42ee-bed6-818c57159227-kube-api-access-cf9vp\") pod \"payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk\" (UID: \"51a7e3aa-25a3-42ee-bed6-818c57159227\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk"
Nov 02 13:19:22.607822 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.607791    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk"
Nov 02 13:19:22.739887 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.739855    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np"
Nov 02 13:19:22.992177 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:22.992149    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk"]
Nov 02 13:19:22.993945 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:19:22.993915    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51a7e3aa_25a3_42ee_bed6_818c57159227.slice/crio-474bbaef542cb57963d2c2300fd58d997d20dcc8f793d687d04b8424c1cd908a WatchSource:0}: Error finding container 474bbaef542cb57963d2c2300fd58d997d20dcc8f793d687d04b8424c1cd908a: Status 404 returned error can't find the container with id 474bbaef542cb57963d2c2300fd58d997d20dcc8f793d687d04b8424c1cd908a
Nov 02 13:19:23.120017 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:23.119984    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np"]
Nov 02 13:19:23.121726 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:19:23.121692    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d136b78_8365_465e_851c_962275c7b35e.slice/crio-e3ce3f32c86f6f298e113551bf2e447c41a352ff800d75b2c90ca20f7ce71e2b WatchSource:0}: Error finding container e3ce3f32c86f6f298e113551bf2e447c41a352ff800d75b2c90ca20f7ce71e2b: Status 404 returned error can't find the container with id e3ce3f32c86f6f298e113551bf2e447c41a352ff800d75b2c90ca20f7ce71e2b
Nov 02 13:19:23.208875 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:23.208832    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk" event={"ID":"51a7e3aa-25a3-42ee-bed6-818c57159227","Type":"ContainerStarted","Data":"cf436e1fa5feb1c20e5d43b3fe0c266072dbd534002e4366d83ac914d5de45bf"}
Nov 02 13:19:23.209051 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:23.208896    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk" event={"ID":"51a7e3aa-25a3-42ee-bed6-818c57159227","Type":"ContainerStarted","Data":"474bbaef542cb57963d2c2300fd58d997d20dcc8f793d687d04b8424c1cd908a"}
Nov 02 13:19:23.209051 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:23.208924    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk"
Nov 02 13:19:23.210421 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:23.210397    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np" event={"ID":"7d136b78-8365-465e-851c-962275c7b35e","Type":"ContainerStarted","Data":"b5025bea77fd84fed9e7a338add551f6cb6f85261a69c42941847fa19a241c68"}
Nov 02 13:19:23.210519 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:23.210438    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np" event={"ID":"7d136b78-8365-465e-851c-962275c7b35e","Type":"ContainerStarted","Data":"e3ce3f32c86f6f298e113551bf2e447c41a352ff800d75b2c90ca20f7ce71e2b"}
Nov 02 13:19:23.210519 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:23.210466    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np"
Nov 02 13:19:23.329233 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:23.329126    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk" podStartSLOduration=1.329111468 podStartE2EDuration="1.329111468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:19:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:19:23.32668763 +0000 UTC m=+4989.898037390" watchObservedRunningTime="2026-11-02 13:19:23.329111468 +0000 UTC m=+4989.900461225"
Nov 02 13:19:23.449101 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:23.448275    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np" podStartSLOduration=1.448255722 podStartE2EDuration="1.448255722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:19:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:19:23.444933928 +0000 UTC m=+4990.016283685" watchObservedRunningTime="2026-11-02 13:19:23.448255722 +0000 UTC m=+4990.019605480"
Nov 02 13:19:24.964269 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:24.964182    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"]
Nov 02 13:19:24.968069 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:24.968046    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:19:25.019953 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:25.019917    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjj66\" (UniqueName: \"kubernetes.io/projected/5ac4fa47-99b7-4935-a9f1-a756a299d861-kube-api-access-jjj66\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7\" (UID: \"5ac4fa47-99b7-4935-a9f1-a756a299d861\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:19:25.020123 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:25.019959    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ac4fa47-99b7-4935-a9f1-a756a299d861-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7\" (UID: \"5ac4fa47-99b7-4935-a9f1-a756a299d861\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:19:25.020123 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:25.019976    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ac4fa47-99b7-4935-a9f1-a756a299d861-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7\" (UID: \"5ac4fa47-99b7-4935-a9f1-a756a299d861\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:19:25.020225 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:25.020145    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ac4fa47-99b7-4935-a9f1-a756a299d861-metrics-certs\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7\" (UID: \"5ac4fa47-99b7-4935-a9f1-a756a299d861\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:19:25.071438 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:25.071402    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-62ca10-serving-cert\""
Nov 02 13:19:25.071608 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:25.071445    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pl94d\""
Nov 02 13:19:25.071677 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:25.071657    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 02 13:19:25.074443 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:25.074418    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"]
Nov 02 13:19:25.121415 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:25.121377    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ac4fa47-99b7-4935-a9f1-a756a299d861-metrics-certs\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7\" (UID: \"5ac4fa47-99b7-4935-a9f1-a756a299d861\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:19:25.121643 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:25.121508    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjj66\" (UniqueName: \"kubernetes.io/projected/5ac4fa47-99b7-4935-a9f1-a756a299d861-kube-api-access-jjj66\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7\" (UID: \"5ac4fa47-99b7-4935-a9f1-a756a299d861\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:19:25.121643 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:25.121544    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ac4fa47-99b7-4935-a9f1-a756a299d861-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7\" (UID: \"5ac4fa47-99b7-4935-a9f1-a756a299d861\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:19:25.121643 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:25.121569    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ac4fa47-99b7-4935-a9f1-a756a299d861-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7\" (UID: \"5ac4fa47-99b7-4935-a9f1-a756a299d861\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:19:25.122025 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:19:25.121988    4942 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-serving-cert: secret "maas-api-e2e-worker-single-main-62ca10-serving-cert" not found
Nov 02 13:19:25.122150 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:19:25.122061    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ac4fa47-99b7-4935-a9f1-a756a299d861-maas-api-tls podName:5ac4fa47-99b7-4935-a9f1-a756a299d861 nodeName:}" failed. No retries permitted until 2026-11-02 13:19:25.622042599 +0000 UTC m=+4992.193392336 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5ac4fa47-99b7-4935-a9f1-a756a299d861-maas-api-tls") pod "maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7" (UID: "5ac4fa47-99b7-4935-a9f1-a756a299d861") : secret "maas-api-e2e-worker-single-main-62ca10-serving-cert" not found
Nov 02 13:19:25.122440 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:25.122416    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ac4fa47-99b7-4935-a9f1-a756a299d861-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7\" (UID: \"5ac4fa47-99b7-4935-a9f1-a756a299d861\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:19:25.124247 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:25.124220    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ac4fa47-99b7-4935-a9f1-a756a299d861-metrics-certs\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7\" (UID: \"5ac4fa47-99b7-4935-a9f1-a756a299d861\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:19:25.190860 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:25.190828    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjj66\" (UniqueName: \"kubernetes.io/projected/5ac4fa47-99b7-4935-a9f1-a756a299d861-kube-api-access-jjj66\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7\" (UID: \"5ac4fa47-99b7-4935-a9f1-a756a299d861\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:19:25.626742 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:25.626702    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ac4fa47-99b7-4935-a9f1-a756a299d861-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7\" (UID: \"5ac4fa47-99b7-4935-a9f1-a756a299d861\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:19:25.629260 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:25.629233    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ac4fa47-99b7-4935-a9f1-a756a299d861-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7\" (UID: \"5ac4fa47-99b7-4935-a9f1-a756a299d861\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:19:25.882593 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:25.882507    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:19:26.263707 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:26.263675    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"]
Nov 02 13:19:26.264430 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:19:26.264405    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ac4fa47_99b7_4935_a9f1_a756a299d861.slice/crio-24e39c9882986ca976211e336d457abc7819709d4957ace2fa5ecc576d68ddd8 WatchSource:0}: Error finding container 24e39c9882986ca976211e336d457abc7819709d4957ace2fa5ecc576d68ddd8: Status 404 returned error can't find the container with id 24e39c9882986ca976211e336d457abc7819709d4957ace2fa5ecc576d68ddd8
Nov 02 13:19:26.265813 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:26.265793    4942 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 13:19:27.237148 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:27.237118    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7" event={"ID":"5ac4fa47-99b7-4935-a9f1-a756a299d861","Type":"ContainerStarted","Data":"24e39c9882986ca976211e336d457abc7819709d4957ace2fa5ecc576d68ddd8"}
Nov 02 13:19:28.243775 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:28.243729    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7" event={"ID":"5ac4fa47-99b7-4935-a9f1-a756a299d861","Type":"ContainerStarted","Data":"96cb102e48ca139f93001ba96c183572a593c42a33d2efad42595f4f475db88b"}
Nov 02 13:19:28.244257 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:28.243887    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:19:28.370259 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:28.370204    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7" podStartSLOduration=3.458984028 podStartE2EDuration="4.370189653s" totalImagesPullingTime="911.205625ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:19:28.368032829 +0000 UTC m=+4994.939382587" watchObservedRunningTime="2026-11-02 13:19:28.370189653 +0000 UTC m=+4994.941539410"
Nov 02 13:19:30.982715 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:19:30.982677    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:19:34.218793 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:34.218761    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np"
Nov 02 13:19:34.219409 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:34.218837    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk"
Nov 02 13:19:34.284606 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:34.284569    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:19:43.352401 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:43.352356    4942 generic.go:356] "Generic (PLEG): container finished" podID="d0ffdc12-01f9-420e-95de-d980a47b4334" containerID="b1fcc0a591bcb67169c550b1e49bf64d06431d278a36806a11eddea2a7fdcdc6" exitCode=0
Nov 02 13:19:43.352958 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:43.352435    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6" event={"ID":"d0ffdc12-01f9-420e-95de-d980a47b4334","Type":"ContainerDied","Data":"b1fcc0a591bcb67169c550b1e49bf64d06431d278a36806a11eddea2a7fdcdc6"}
Nov 02 13:19:43.353038 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:43.353012    4942 scope.go:122] "RemoveContainer" containerID="b1fcc0a591bcb67169c550b1e49bf64d06431d278a36806a11eddea2a7fdcdc6"
Nov 02 13:19:43.354214 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:43.354183    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:19:43.354284 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:43.354269    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:19:43.354346 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:43.354330    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:19:44.382677 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:44.382642    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6" event={"ID":"d0ffdc12-01f9-420e-95de-d980a47b4334","Type":"ContainerStarted","Data":"b0c856705f6000eb2d780a1db864d98dfdc19a3096b5e15633646d150148f2d6"}
Nov 02 13:19:45.374322 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:45.373657    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:45.377321 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:45.375684    4942 patch_prober.go:29] interesting pod/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Nov 02 13:19:45.377321 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:45.375754    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6" podUID="d0ffdc12-01f9-420e-95de-d980a47b4334" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Nov 02 13:19:46.123616 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:46.123579    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:46.366730 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:46.366686    4942 patch_prober.go:29] interesting pod/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Nov 02 13:19:46.367023 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:46.366990    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6" podUID="d0ffdc12-01f9-420e-95de-d980a47b4334" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Nov 02 13:19:47.373321 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:47.372679    4942 patch_prober.go:29] interesting pod/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Nov 02 13:19:47.373321 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:47.372740    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6" podUID="d0ffdc12-01f9-420e-95de-d980a47b4334" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Nov 02 13:19:48.372324 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:48.368667    4942 patch_prober.go:29] interesting pod/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Nov 02 13:19:48.372324 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:48.368732    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6" podUID="d0ffdc12-01f9-420e-95de-d980a47b4334" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Nov 02 13:19:49.361268 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:49.361224    4942 patch_prober.go:29] interesting pod/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body=
Nov 02 13:19:49.361787 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:49.361326    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6" podUID="d0ffdc12-01f9-420e-95de-d980a47b4334" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused"
Nov 02 13:19:50.364917 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:50.364886    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:19:50.366048 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:19:50.366023    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:20:00.984595 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:20:00.984559    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:20:30.982417 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:20:30.982380    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:20:55.551130 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:55.551036    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"]
Nov 02 13:20:55.551560 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:55.551252    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7" podUID="5ac4fa47-99b7-4935-a9f1-a756a299d861" containerName="maas-api" containerID="cri-o://96cb102e48ca139f93001ba96c183572a593c42a33d2efad42595f4f475db88b" gracePeriod=30
Nov 02 13:20:55.587056 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:55.587017    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk"]
Nov 02 13:20:55.705748 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:55.705715    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np"]
Nov 02 13:20:56.793183 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:56.793153    4942 generic.go:356] "Generic (PLEG): container finished" podID="5ac4fa47-99b7-4935-a9f1-a756a299d861" containerID="96cb102e48ca139f93001ba96c183572a593c42a33d2efad42595f4f475db88b" exitCode=0
Nov 02 13:20:56.793697 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:56.793221    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7" event={"ID":"5ac4fa47-99b7-4935-a9f1-a756a299d861","Type":"ContainerDied","Data":"96cb102e48ca139f93001ba96c183572a593c42a33d2efad42595f4f475db88b"}
Nov 02 13:20:56.826901 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:56.826874    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:20:56.886852 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:56.886826    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ac4fa47-99b7-4935-a9f1-a756a299d861-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ac4fa47-99b7-4935-a9f1-a756a299d861-openshift-service-ca\") pod \"5ac4fa47-99b7-4935-a9f1-a756a299d861\" (UID: \"5ac4fa47-99b7-4935-a9f1-a756a299d861\") "
Nov 02 13:20:56.886995 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:56.886886    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5ac4fa47-99b7-4935-a9f1-a756a299d861-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ac4fa47-99b7-4935-a9f1-a756a299d861-maas-api-tls\") pod \"5ac4fa47-99b7-4935-a9f1-a756a299d861\" (UID: \"5ac4fa47-99b7-4935-a9f1-a756a299d861\") "
Nov 02 13:20:56.886995 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:56.886905    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5ac4fa47-99b7-4935-a9f1-a756a299d861-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ac4fa47-99b7-4935-a9f1-a756a299d861-metrics-certs\") pod \"5ac4fa47-99b7-4935-a9f1-a756a299d861\" (UID: \"5ac4fa47-99b7-4935-a9f1-a756a299d861\") "
Nov 02 13:20:56.887135 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:56.887021    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ac4fa47-99b7-4935-a9f1-a756a299d861-kube-api-access-jjj66\" (UniqueName: \"kubernetes.io/projected/5ac4fa47-99b7-4935-a9f1-a756a299d861-kube-api-access-jjj66\") pod \"5ac4fa47-99b7-4935-a9f1-a756a299d861\" (UID: \"5ac4fa47-99b7-4935-a9f1-a756a299d861\") "
Nov 02 13:20:56.887217 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:56.887183    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ac4fa47-99b7-4935-a9f1-a756a299d861-openshift-service-ca" pod "5ac4fa47-99b7-4935-a9f1-a756a299d861" (UID: "5ac4fa47-99b7-4935-a9f1-a756a299d861"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:20:56.887412 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:56.887393    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ac4fa47-99b7-4935-a9f1-a756a299d861-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:20:56.888989 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:56.888961    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ac4fa47-99b7-4935-a9f1-a756a299d861-maas-api-tls" pod "5ac4fa47-99b7-4935-a9f1-a756a299d861" (UID: "5ac4fa47-99b7-4935-a9f1-a756a299d861"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:20:56.889114 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:56.889089    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ac4fa47-99b7-4935-a9f1-a756a299d861-metrics-certs" pod "5ac4fa47-99b7-4935-a9f1-a756a299d861" (UID: "5ac4fa47-99b7-4935-a9f1-a756a299d861"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:20:56.889169 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:56.889115    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ac4fa47-99b7-4935-a9f1-a756a299d861-kube-api-access-jjj66" pod "5ac4fa47-99b7-4935-a9f1-a756a299d861" (UID: "5ac4fa47-99b7-4935-a9f1-a756a299d861"). InnerVolumeSpecName "kube-api-access-jjj66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:20:56.988617 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:56.988539    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ac4fa47-99b7-4935-a9f1-a756a299d861-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:20:56.988617 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:56.988567    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ac4fa47-99b7-4935-a9f1-a756a299d861-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:20:56.988617 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:56.988581    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjj66\" (UniqueName: \"kubernetes.io/projected/5ac4fa47-99b7-4935-a9f1-a756a299d861-kube-api-access-jjj66\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:20:57.801078 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:57.800990    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"
Nov 02 13:20:57.801078 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:57.801007    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7" event={"ID":"5ac4fa47-99b7-4935-a9f1-a756a299d861","Type":"ContainerDied","Data":"24e39c9882986ca976211e336d457abc7819709d4957ace2fa5ecc576d68ddd8"}
Nov 02 13:20:57.801078 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:57.801071    4942 scope.go:122] "RemoveContainer" containerID="96cb102e48ca139f93001ba96c183572a593c42a33d2efad42595f4f475db88b"
Nov 02 13:20:57.965567 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:57.965534    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"]
Nov 02 13:20:57.969804 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:57.969779    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-sqwr7"]
Nov 02 13:20:59.544210 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:20:59.544176    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ac4fa47-99b7-4935-a9f1-a756a299d861" path="/var/lib/kubelet/pods/5ac4fa47-99b7-4935-a9f1-a756a299d861/volumes"
Nov 02 13:21:00.588051 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:00.588001    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk" podUID="51a7e3aa-25a3-42ee-bed6-818c57159227" containerName="payload-processing" containerID="cri-o://cf436e1fa5feb1c20e5d43b3fe0c266072dbd534002e4366d83ac914d5de45bf" gracePeriod=25
Nov 02 13:21:00.706430 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:00.706379    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np" podUID="7d136b78-8365-465e-851c-962275c7b35e" containerName="payload-pre-processing" containerID="cri-o://b5025bea77fd84fed9e7a338add551f6cb6f85261a69c42941847fa19a241c68" gracePeriod=25
Nov 02 13:21:00.982995 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:21:00.982964    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:21:04.218161 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:04.218115    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np" podUID="7d136b78-8365-465e-851c-962275c7b35e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 13:21:04.218716 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:04.218213    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk" podUID="51a7e3aa-25a3-42ee-bed6-818c57159227" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 13:21:14.217918 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:14.217881    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np" podUID="7d136b78-8365-465e-851c-962275c7b35e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 13:21:14.218329 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:14.217890    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk" podUID="51a7e3aa-25a3-42ee-bed6-818c57159227" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 13:21:14.248618 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:14.248591    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:21:14.266050 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:14.266030    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:21:20.878876 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:20.878852    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk"
Nov 02 13:21:20.914872 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:20.914837    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51a7e3aa-25a3-42ee-bed6-818c57159227-kube-api-access-cf9vp\" (UniqueName: \"kubernetes.io/projected/51a7e3aa-25a3-42ee-bed6-818c57159227-kube-api-access-cf9vp\") pod \"51a7e3aa-25a3-42ee-bed6-818c57159227\" (UID: \"51a7e3aa-25a3-42ee-bed6-818c57159227\") "
Nov 02 13:21:20.915039 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:20.914985    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51a7e3aa-25a3-42ee-bed6-818c57159227-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51a7e3aa-25a3-42ee-bed6-818c57159227-plugins-config-volume\") pod \"51a7e3aa-25a3-42ee-bed6-818c57159227\" (UID: \"51a7e3aa-25a3-42ee-bed6-818c57159227\") "
Nov 02 13:21:20.915470 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:20.915434    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51a7e3aa-25a3-42ee-bed6-818c57159227-plugins-config-volume" pod "51a7e3aa-25a3-42ee-bed6-818c57159227" (UID: "51a7e3aa-25a3-42ee-bed6-818c57159227"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:21:20.918359 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:20.918332    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51a7e3aa-25a3-42ee-bed6-818c57159227-kube-api-access-cf9vp" pod "51a7e3aa-25a3-42ee-bed6-818c57159227" (UID: "51a7e3aa-25a3-42ee-bed6-818c57159227"). InnerVolumeSpecName "kube-api-access-cf9vp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:21:20.936155 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:20.936130    4942 generic.go:356] "Generic (PLEG): container finished" podID="7d136b78-8365-465e-851c-962275c7b35e" containerID="b5025bea77fd84fed9e7a338add551f6cb6f85261a69c42941847fa19a241c68" exitCode=0
Nov 02 13:21:20.936285 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:20.936201    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np" event={"ID":"7d136b78-8365-465e-851c-962275c7b35e","Type":"ContainerDied","Data":"b5025bea77fd84fed9e7a338add551f6cb6f85261a69c42941847fa19a241c68"}
Nov 02 13:21:20.937632 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:20.937611    4942 generic.go:356] "Generic (PLEG): container finished" podID="51a7e3aa-25a3-42ee-bed6-818c57159227" containerID="cf436e1fa5feb1c20e5d43b3fe0c266072dbd534002e4366d83ac914d5de45bf" exitCode=0
Nov 02 13:21:20.937750 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:20.937657    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk"
Nov 02 13:21:20.937750 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:20.937685    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk" event={"ID":"51a7e3aa-25a3-42ee-bed6-818c57159227","Type":"ContainerDied","Data":"cf436e1fa5feb1c20e5d43b3fe0c266072dbd534002e4366d83ac914d5de45bf"}
Nov 02 13:21:20.937750 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:20.937730    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk" event={"ID":"51a7e3aa-25a3-42ee-bed6-818c57159227","Type":"ContainerDied","Data":"474bbaef542cb57963d2c2300fd58d997d20dcc8f793d687d04b8424c1cd908a"}
Nov 02 13:21:20.937909 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:20.937752    4942 scope.go:122] "RemoveContainer" containerID="cf436e1fa5feb1c20e5d43b3fe0c266072dbd534002e4366d83ac914d5de45bf"
Nov 02 13:21:20.947925 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:20.947902    4942 scope.go:122] "RemoveContainer" containerID="cf436e1fa5feb1c20e5d43b3fe0c266072dbd534002e4366d83ac914d5de45bf"
Nov 02 13:21:20.948251 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:21:20.948226    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf436e1fa5feb1c20e5d43b3fe0c266072dbd534002e4366d83ac914d5de45bf\": container with ID starting with cf436e1fa5feb1c20e5d43b3fe0c266072dbd534002e4366d83ac914d5de45bf not found: ID does not exist" containerID="cf436e1fa5feb1c20e5d43b3fe0c266072dbd534002e4366d83ac914d5de45bf"
Nov 02 13:21:20.948910 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:20.948258    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf436e1fa5feb1c20e5d43b3fe0c266072dbd534002e4366d83ac914d5de45bf"} err="failed to get container status \"cf436e1fa5feb1c20e5d43b3fe0c266072dbd534002e4366d83ac914d5de45bf\": rpc error: code = NotFound desc = could not find container \"cf436e1fa5feb1c20e5d43b3fe0c266072dbd534002e4366d83ac914d5de45bf\": container with ID starting with cf436e1fa5feb1c20e5d43b3fe0c266072dbd534002e4366d83ac914d5de45bf not found: ID does not exist"
Nov 02 13:21:20.960461 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:20.960442    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np"
Nov 02 13:21:21.015680 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:21.015658    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d136b78-8365-465e-851c-962275c7b35e-kube-api-access-h9v4k\" (UniqueName: \"kubernetes.io/projected/7d136b78-8365-465e-851c-962275c7b35e-kube-api-access-h9v4k\") pod \"7d136b78-8365-465e-851c-962275c7b35e\" (UID: \"7d136b78-8365-465e-851c-962275c7b35e\") "
Nov 02 13:21:21.015800 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:21.015693    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d136b78-8365-465e-851c-962275c7b35e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d136b78-8365-465e-851c-962275c7b35e-plugins-config-volume\") pod \"7d136b78-8365-465e-851c-962275c7b35e\" (UID: \"7d136b78-8365-465e-851c-962275c7b35e\") "
Nov 02 13:21:21.015886 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:21.015873    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cf9vp\" (UniqueName: \"kubernetes.io/projected/51a7e3aa-25a3-42ee-bed6-818c57159227-kube-api-access-cf9vp\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:21:21.015948 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:21.015888    4942 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51a7e3aa-25a3-42ee-bed6-818c57159227-plugins-config-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:21:21.016134 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:21.016114    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d136b78-8365-465e-851c-962275c7b35e-plugins-config-volume" pod "7d136b78-8365-465e-851c-962275c7b35e" (UID: "7d136b78-8365-465e-851c-962275c7b35e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:21:21.017808 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:21.017786    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d136b78-8365-465e-851c-962275c7b35e-kube-api-access-h9v4k" pod "7d136b78-8365-465e-851c-962275c7b35e" (UID: "7d136b78-8365-465e-851c-962275c7b35e"). InnerVolumeSpecName "kube-api-access-h9v4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:21:21.116656 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:21.116588    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9v4k\" (UniqueName: \"kubernetes.io/projected/7d136b78-8365-465e-851c-962275c7b35e-kube-api-access-h9v4k\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:21:21.116656 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:21.116622    4942 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d136b78-8365-465e-851c-962275c7b35e-plugins-config-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:21:21.223882 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:21.223853    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk"]
Nov 02 13:21:21.230161 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:21.230128    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-62ca10-5bd99fd49l7vlk"]
Nov 02 13:21:21.542474 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:21.542382    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51a7e3aa-25a3-42ee-bed6-818c57159227" path="/var/lib/kubelet/pods/51a7e3aa-25a3-42ee-bed6-818c57159227/volumes"
Nov 02 13:21:21.944997 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:21.944961    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np" event={"ID":"7d136b78-8365-465e-851c-962275c7b35e","Type":"ContainerDied","Data":"e3ce3f32c86f6f298e113551bf2e447c41a352ff800d75b2c90ca20f7ce71e2b"}
Nov 02 13:21:21.944997 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:21.944994    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np"
Nov 02 13:21:21.945504 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:21.945024    4942 scope.go:122] "RemoveContainer" containerID="b5025bea77fd84fed9e7a338add551f6cb6f85261a69c42941847fa19a241c68"
Nov 02 13:21:22.110249 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:22.110221    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np"]
Nov 02 13:21:22.159757 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:22.159725    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-62ca10-5ffff9p6np"]
Nov 02 13:21:23.542384 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:23.542348    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d136b78-8365-465e-851c-962275c7b35e" path="/var/lib/kubelet/pods/7d136b78-8365-465e-851c-962275c7b35e/volumes"
Nov 02 13:21:25.051328 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:25.051285    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8"]
Nov 02 13:21:25.100853 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:25.100796    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f904bff9-zx5t8"]
Nov 02 13:21:25.541974 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:25.541937    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2982eabd-924c-4f43-81bf-bc39e197a79c" path="/var/lib/kubelet/pods/2982eabd-924c-4f43-81bf-bc39e197a79c/volumes"
Nov 02 13:21:30.982101 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:21:30.982059    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:21:32.444745 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.444711    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"]
Nov 02 13:21:32.445138 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.445129    4942 cpu_manager.go:414] "containerMap: removing container" podUID="51a7e3aa-25a3-42ee-bed6-818c57159227" containerName="payload-processing"
Nov 02 13:21:32.445180 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.445140    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51a7e3aa-25a3-42ee-bed6-818c57159227" containerName="payload-processing"
Nov 02 13:21:32.445180 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.445150    4942 cpu_manager.go:414] "containerMap: removing container" podUID="5ac4fa47-99b7-4935-a9f1-a756a299d861" containerName="maas-api"
Nov 02 13:21:32.445180 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.445155    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ac4fa47-99b7-4935-a9f1-a756a299d861" containerName="maas-api"
Nov 02 13:21:32.445284 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.445185    4942 cpu_manager.go:414] "containerMap: removing container" podUID="7d136b78-8365-465e-851c-962275c7b35e" containerName="payload-pre-processing"
Nov 02 13:21:32.445284 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.445191    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d136b78-8365-465e-851c-962275c7b35e" containerName="payload-pre-processing"
Nov 02 13:21:32.445284 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.445253    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d136b78-8365-465e-851c-962275c7b35e" containerName="payload-pre-processing"
Nov 02 13:21:32.445284 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.445262    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="51a7e3aa-25a3-42ee-bed6-818c57159227" containerName="payload-processing"
Nov 02 13:21:32.445284 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.445269    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ac4fa47-99b7-4935-a9f1-a756a299d861" containerName="maas-api"
Nov 02 13:21:32.448426 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.448409    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:21:32.520378 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.520341    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44\" (UID: \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:21:32.520543 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.520393    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6885\" (UniqueName: \"kubernetes.io/projected/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-kube-api-access-c6885\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44\" (UID: \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:21:32.520543 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.520494    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-metrics-certs\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44\" (UID: \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:21:32.520633 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.520600    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44\" (UID: \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:21:32.549906 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.549858    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-62ca10-serving-cert\""
Nov 02 13:21:32.549906 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.549876    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pl94d\""
Nov 02 13:21:32.550079 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.549951    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 02 13:21:32.552579 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.552562    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"]
Nov 02 13:21:32.621359 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.621330    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44\" (UID: \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:21:32.621539 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.621362    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6885\" (UniqueName: \"kubernetes.io/projected/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-kube-api-access-c6885\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44\" (UID: \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:21:32.621539 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.621398    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-metrics-certs\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44\" (UID: \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:21:32.621539 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.621449    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44\" (UID: \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:21:32.621715 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:21:32.621580    4942 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-serving-cert: secret "maas-api-e2e-worker-single-main-62ca10-serving-cert" not found
Nov 02 13:21:32.621715 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:21:32.621633    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-maas-api-tls podName:5e2bf3a2-74f3-4515-848a-f91e63ac6f45 nodeName:}" failed. No retries permitted until 2026-11-02 13:21:33.121613285 +0000 UTC m=+5119.692963021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-maas-api-tls") pod "maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44" (UID: "5e2bf3a2-74f3-4515-848a-f91e63ac6f45") : secret "maas-api-e2e-worker-single-main-62ca10-serving-cert" not found
Nov 02 13:21:32.622114 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.622091    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44\" (UID: \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:21:32.624085 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.624056    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-metrics-certs\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44\" (UID: \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:21:32.684565 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:32.684541    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6885\" (UniqueName: \"kubernetes.io/projected/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-kube-api-access-c6885\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44\" (UID: \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:21:33.125711 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:33.125671    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44\" (UID: \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:21:33.128484 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:33.128446    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44\" (UID: \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:21:33.362417 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:33.362381    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:21:33.740383 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:33.740353    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"]
Nov 02 13:21:33.742033 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:21:33.742005    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e2bf3a2_74f3_4515_848a_f91e63ac6f45.slice/crio-0e6a63af1bd7d022f103168c6c01b4034bd2b7e9a1217c7879af619fe0dcbfe7 WatchSource:0}: Error finding container 0e6a63af1bd7d022f103168c6c01b4034bd2b7e9a1217c7879af619fe0dcbfe7: Status 404 returned error can't find the container with id 0e6a63af1bd7d022f103168c6c01b4034bd2b7e9a1217c7879af619fe0dcbfe7
Nov 02 13:21:34.021884 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:34.021807    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44" event={"ID":"5e2bf3a2-74f3-4515-848a-f91e63ac6f45","Type":"ContainerStarted","Data":"0e6a63af1bd7d022f103168c6c01b4034bd2b7e9a1217c7879af619fe0dcbfe7"}
Nov 02 13:21:35.028783 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:35.028693    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44" event={"ID":"5e2bf3a2-74f3-4515-848a-f91e63ac6f45","Type":"ContainerStarted","Data":"7e13079f09b33fe7389b71d177a30dc18746bfbbf63643681f0a874f4f6bc141"}
Nov 02 13:21:35.029146 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:35.028786    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:21:35.147974 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:35.147906    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44" podStartSLOduration=2.246982094 podStartE2EDuration="3.147889343s" totalImagesPullingTime="900.907249ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:21:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:21:35.147119263 +0000 UTC m=+5121.718469020" watchObservedRunningTime="2026-11-02 13:21:35.147889343 +0000 UTC m=+5121.719239100"
Nov 02 13:21:41.040228 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:21:41.040197    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:22:00.982784 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:22:00.982748    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:22:01.486861 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:01.486820    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"]
Nov 02 13:22:01.487105 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:01.487077    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44" podUID="5e2bf3a2-74f3-4515-848a-f91e63ac6f45" containerName="maas-api" containerID="cri-o://7e13079f09b33fe7389b71d177a30dc18746bfbbf63643681f0a874f4f6bc141" gracePeriod=30
Nov 02 13:22:01.743224 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:01.743166    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:22:01.896042 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:01.896005    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-maas-api-tls\") pod \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\" (UID: \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\") "
Nov 02 13:22:01.896212 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:01.896142    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-kube-api-access-c6885\" (UniqueName: \"kubernetes.io/projected/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-kube-api-access-c6885\") pod \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\" (UID: \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\") "
Nov 02 13:22:01.896212 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:01.896178    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-openshift-service-ca\") pod \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\" (UID: \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\") "
Nov 02 13:22:01.896348 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:01.896217    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-metrics-certs\") pod \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\" (UID: \"5e2bf3a2-74f3-4515-848a-f91e63ac6f45\") "
Nov 02 13:22:01.896603 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:01.896571    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-openshift-service-ca" pod "5e2bf3a2-74f3-4515-848a-f91e63ac6f45" (UID: "5e2bf3a2-74f3-4515-848a-f91e63ac6f45"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:22:01.896725 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:01.896687    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:22:01.898390 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:01.898358    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-maas-api-tls" pod "5e2bf3a2-74f3-4515-848a-f91e63ac6f45" (UID: "5e2bf3a2-74f3-4515-848a-f91e63ac6f45"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:22:01.898570 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:01.898547    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-metrics-certs" pod "5e2bf3a2-74f3-4515-848a-f91e63ac6f45" (UID: "5e2bf3a2-74f3-4515-848a-f91e63ac6f45"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:22:01.902514 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:01.902480    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-kube-api-access-c6885" pod "5e2bf3a2-74f3-4515-848a-f91e63ac6f45" (UID: "5e2bf3a2-74f3-4515-848a-f91e63ac6f45"). InnerVolumeSpecName "kube-api-access-c6885". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:22:01.997948 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:01.997861    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6885\" (UniqueName: \"kubernetes.io/projected/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-kube-api-access-c6885\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:22:01.997948 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:01.997894    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:22:01.997948 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:01.997906    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e2bf3a2-74f3-4515-848a-f91e63ac6f45-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:22:02.189267 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:02.189231    4942 generic.go:356] "Generic (PLEG): container finished" podID="5e2bf3a2-74f3-4515-848a-f91e63ac6f45" containerID="7e13079f09b33fe7389b71d177a30dc18746bfbbf63643681f0a874f4f6bc141" exitCode=0
Nov 02 13:22:02.189471 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:02.189278    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"
Nov 02 13:22:02.189471 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:02.189338    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44" event={"ID":"5e2bf3a2-74f3-4515-848a-f91e63ac6f45","Type":"ContainerDied","Data":"7e13079f09b33fe7389b71d177a30dc18746bfbbf63643681f0a874f4f6bc141"}
Nov 02 13:22:02.189471 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:02.189388    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44" event={"ID":"5e2bf3a2-74f3-4515-848a-f91e63ac6f45","Type":"ContainerDied","Data":"0e6a63af1bd7d022f103168c6c01b4034bd2b7e9a1217c7879af619fe0dcbfe7"}
Nov 02 13:22:02.189471 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:02.189412    4942 scope.go:122] "RemoveContainer" containerID="7e13079f09b33fe7389b71d177a30dc18746bfbbf63643681f0a874f4f6bc141"
Nov 02 13:22:02.199506 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:02.199476    4942 scope.go:122] "RemoveContainer" containerID="7e13079f09b33fe7389b71d177a30dc18746bfbbf63643681f0a874f4f6bc141"
Nov 02 13:22:02.199794 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:22:02.199772    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e13079f09b33fe7389b71d177a30dc18746bfbbf63643681f0a874f4f6bc141\": container with ID starting with 7e13079f09b33fe7389b71d177a30dc18746bfbbf63643681f0a874f4f6bc141 not found: ID does not exist" containerID="7e13079f09b33fe7389b71d177a30dc18746bfbbf63643681f0a874f4f6bc141"
Nov 02 13:22:02.199859 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:02.199803    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e13079f09b33fe7389b71d177a30dc18746bfbbf63643681f0a874f4f6bc141"} err="failed to get container status \"7e13079f09b33fe7389b71d177a30dc18746bfbbf63643681f0a874f4f6bc141\": rpc error: code = NotFound desc = could not find container \"7e13079f09b33fe7389b71d177a30dc18746bfbbf63643681f0a874f4f6bc141\": container with ID starting with 7e13079f09b33fe7389b71d177a30dc18746bfbbf63643681f0a874f4f6bc141 not found: ID does not exist"
Nov 02 13:22:02.354108 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:02.354077    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"]
Nov 02 13:22:02.358308 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:02.358272    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-62ca10-5fff986775-4cz44"]
Nov 02 13:22:03.541732 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:03.541699    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e2bf3a2-74f3-4515-848a-f91e63ac6f45" path="/var/lib/kubelet/pods/5e2bf3a2-74f3-4515-848a-f91e63ac6f45/volumes"
Nov 02 13:22:04.076347 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:04.076315    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"]
Nov 02 13:22:04.076555 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:04.076533    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6" podUID="d0ffdc12-01f9-420e-95de-d980a47b4334" containerName="istio-proxy" containerID="cri-o://b0c856705f6000eb2d780a1db864d98dfdc19a3096b5e15633646d150148f2d6" gracePeriod=30
Nov 02 13:22:05.367742 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:05.367697    4942 patch_prober.go:29] interesting pod/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 13:22:05.368147 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:05.367761    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6" podUID="d0ffdc12-01f9-420e-95de-d980a47b4334" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 13:22:09.235947 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.235914    4942 generic.go:356] "Generic (PLEG): container finished" podID="d0ffdc12-01f9-420e-95de-d980a47b4334" containerID="b0c856705f6000eb2d780a1db864d98dfdc19a3096b5e15633646d150148f2d6" exitCode=0
Nov 02 13:22:09.236357 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.235996    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6" event={"ID":"d0ffdc12-01f9-420e-95de-d980a47b4334","Type":"ContainerDied","Data":"b0c856705f6000eb2d780a1db864d98dfdc19a3096b5e15633646d150148f2d6"}
Nov 02 13:22:09.236357 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.236075    4942 scope.go:122] "RemoveContainer" containerID="b1fcc0a591bcb67169c550b1e49bf64d06431d278a36806a11eddea2a7fdcdc6"
Nov 02 13:22:09.344400 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.344342    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:22:09.369267 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.369239    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-token\" (UniqueName: \"kubernetes.io/projected/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-token\") pod \"d0ffdc12-01f9-420e-95de-d980a47b4334\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") "
Nov 02 13:22:09.369444 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.369274    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-credential-socket\") pod \"d0ffdc12-01f9-420e-95de-d980a47b4334\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") "
Nov 02 13:22:09.369444 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.369316    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-envoy\") pod \"d0ffdc12-01f9-420e-95de-d980a47b4334\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") "
Nov 02 13:22:09.369444 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.369340    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-workload-certs\") pod \"d0ffdc12-01f9-420e-95de-d980a47b4334\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") "
Nov 02 13:22:09.369444 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.369388    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-data\") pod \"d0ffdc12-01f9-420e-95de-d980a47b4334\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") "
Nov 02 13:22:09.369444 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.369410    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0ffdc12-01f9-420e-95de-d980a47b4334-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0ffdc12-01f9-420e-95de-d980a47b4334-istiod-ca-cert\") pod \"d0ffdc12-01f9-420e-95de-d980a47b4334\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") "
Nov 02 13:22:09.369712 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.369473    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0ffdc12-01f9-420e-95de-d980a47b4334-kube-api-access-glsrv\" (UniqueName: \"kubernetes.io/projected/d0ffdc12-01f9-420e-95de-d980a47b4334-kube-api-access-glsrv\") pod \"d0ffdc12-01f9-420e-95de-d980a47b4334\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") "
Nov 02 13:22:09.369712 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.369513    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-podinfo\") pod \"d0ffdc12-01f9-420e-95de-d980a47b4334\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") "
Nov 02 13:22:09.369712 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.369568    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-workload-socket\") pod \"d0ffdc12-01f9-420e-95de-d980a47b4334\" (UID: \"d0ffdc12-01f9-420e-95de-d980a47b4334\") "
Nov 02 13:22:09.369712 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.369656    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-credential-socket" pod "d0ffdc12-01f9-420e-95de-d980a47b4334" (UID: "d0ffdc12-01f9-420e-95de-d980a47b4334"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:22:09.370027 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.369957    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-data" pod "d0ffdc12-01f9-420e-95de-d980a47b4334" (UID: "d0ffdc12-01f9-420e-95de-d980a47b4334"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:22:09.370094 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.370062    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-workload-socket" pod "d0ffdc12-01f9-420e-95de-d980a47b4334" (UID: "d0ffdc12-01f9-420e-95de-d980a47b4334"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:22:09.370149 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.370107    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:22:09.370149 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.370124    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:22:09.370149 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.370126    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0ffdc12-01f9-420e-95de-d980a47b4334-istiod-ca-cert" pod "d0ffdc12-01f9-420e-95de-d980a47b4334" (UID: "d0ffdc12-01f9-420e-95de-d980a47b4334"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:22:09.370326 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.370145    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-workload-certs" pod "d0ffdc12-01f9-420e-95de-d980a47b4334" (UID: "d0ffdc12-01f9-420e-95de-d980a47b4334"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:22:09.371699 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.371673    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-token" pod "d0ffdc12-01f9-420e-95de-d980a47b4334" (UID: "d0ffdc12-01f9-420e-95de-d980a47b4334"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:22:09.371927 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.371894    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-envoy" pod "d0ffdc12-01f9-420e-95de-d980a47b4334" (UID: "d0ffdc12-01f9-420e-95de-d980a47b4334"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:22:09.372352 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.372266    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-podinfo" pod "d0ffdc12-01f9-420e-95de-d980a47b4334" (UID: "d0ffdc12-01f9-420e-95de-d980a47b4334"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 13:22:09.372352 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.372317    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ffdc12-01f9-420e-95de-d980a47b4334-kube-api-access-glsrv" pod "d0ffdc12-01f9-420e-95de-d980a47b4334" (UID: "d0ffdc12-01f9-420e-95de-d980a47b4334"). InnerVolumeSpecName "kube-api-access-glsrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:22:09.470768 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.470742    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glsrv\" (UniqueName: \"kubernetes.io/projected/d0ffdc12-01f9-420e-95de-d980a47b4334-kube-api-access-glsrv\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:22:09.470768 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.470767    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:22:09.470934 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.470778    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:22:09.470934 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.470787    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:22:09.470934 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.470796    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:22:09.470934 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.470806    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0ffdc12-01f9-420e-95de-d980a47b4334-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:22:09.470934 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:09.470817    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0ffdc12-01f9-420e-95de-d980a47b4334-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:22:10.242525 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:10.242484    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"
Nov 02 13:22:10.242525 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:10.242504    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6" event={"ID":"d0ffdc12-01f9-420e-95de-d980a47b4334","Type":"ContainerDied","Data":"42006b03fad9d08003a944bf5d2ec2f37ead55a93157277f948748f265b3ea48"}
Nov 02 13:22:10.243189 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:10.242614    4942 scope.go:122] "RemoveContainer" containerID="b0c856705f6000eb2d780a1db864d98dfdc19a3096b5e15633646d150148f2d6"
Nov 02 13:22:10.407794 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:10.407758    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"]
Nov 02 13:22:10.457227 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:10.457194    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-62ca10-openshift-default-659b69975cbxnn6"]
Nov 02 13:22:11.003892 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.003851    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"]
Nov 02 13:22:11.004365 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.004343    4942 cpu_manager.go:414] "containerMap: removing container" podUID="d0ffdc12-01f9-420e-95de-d980a47b4334" containerName="istio-proxy"
Nov 02 13:22:11.004482 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.004367    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0ffdc12-01f9-420e-95de-d980a47b4334" containerName="istio-proxy"
Nov 02 13:22:11.004482 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.004423    4942 cpu_manager.go:414] "containerMap: removing container" podUID="5e2bf3a2-74f3-4515-848a-f91e63ac6f45" containerName="maas-api"
Nov 02 13:22:11.004482 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.004432    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e2bf3a2-74f3-4515-848a-f91e63ac6f45" containerName="maas-api"
Nov 02 13:22:11.004638 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.004539    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e2bf3a2-74f3-4515-848a-f91e63ac6f45" containerName="maas-api"
Nov 02 13:22:11.004638 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.004555    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0ffdc12-01f9-420e-95de-d980a47b4334" containerName="istio-proxy"
Nov 02 13:22:11.004638 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.004566    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0ffdc12-01f9-420e-95de-d980a47b4334" containerName="istio-proxy"
Nov 02 13:22:11.009105 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.009084    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.063366 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.063332    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-128a019f-kserve-self-signed-certs\""
Nov 02 13:22:11.066143 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.066117    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"]
Nov 02 13:22:11.085834 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.085811    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-home\") pod \"e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") " pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.085940 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.085874    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-model-cache\") pod \"e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") " pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.085983 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.085935    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d2ca598-70b0-442d-a6d9-664ce9c28cca-tls-certs\") pod \"e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") " pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.086054 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.086034    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swmsh\" (UniqueName: \"kubernetes.io/projected/3d2ca598-70b0-442d-a6d9-664ce9c28cca-kube-api-access-swmsh\") pod \"e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") " pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.086098 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.086069    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-tmp-dir\") pod \"e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") " pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.086098 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.086092    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-dshm\") pod \"e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") " pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.186875 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.186840    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-model-cache\") pod \"e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") " pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.186875 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.186878    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d2ca598-70b0-442d-a6d9-664ce9c28cca-tls-certs\") pod \"e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") " pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.187079 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.186921    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swmsh\" (UniqueName: \"kubernetes.io/projected/3d2ca598-70b0-442d-a6d9-664ce9c28cca-kube-api-access-swmsh\") pod \"e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") " pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.187079 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.186942    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-tmp-dir\") pod \"e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") " pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.187079 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.186965    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-dshm\") pod \"e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") " pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.187079 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.187031    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-home\") pod \"e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") " pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.187356 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.187332    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-model-cache\") pod \"e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") " pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.187407 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.187367    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-tmp-dir\") pod \"e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") " pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.187476 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.187410    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-home\") pod \"e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") " pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.189223 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.189199    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-dshm\") pod \"e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") " pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.189570 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.189550    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d2ca598-70b0-442d-a6d9-664ce9c28cca-tls-certs\") pod \"e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") " pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.248048 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.248023    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swmsh\" (UniqueName: \"kubernetes.io/projected/3d2ca598-70b0-442d-a6d9-664ce9c28cca-kube-api-access-swmsh\") pod \"e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") " pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.321030 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.320960    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:11.542548 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.542519    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0ffdc12-01f9-420e-95de-d980a47b4334" path="/var/lib/kubelet/pods/d0ffdc12-01f9-420e-95de-d980a47b4334/volumes"
Nov 02 13:22:11.714874 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:11.714845    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"]
Nov 02 13:22:11.715005 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:22:11.714980    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d2ca598_70b0_442d_a6d9_664ce9c28cca.slice/crio-b3e85491b4d29536a22f9347cb55284df08bc0688a5788841e0a64761deaa099 WatchSource:0}: Error finding container b3e85491b4d29536a22f9347cb55284df08bc0688a5788841e0a64761deaa099: Status 404 returned error can't find the container with id b3e85491b4d29536a22f9347cb55284df08bc0688a5788841e0a64761deaa099
Nov 02 13:22:12.257836 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:12.257795    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg" event={"ID":"3d2ca598-70b0-442d-a6d9-664ce9c28cca","Type":"ContainerStarted","Data":"198473588eb57f77d17d718957012cec6285b7b07c98b2566adc5a194d292d66"}
Nov 02 13:22:12.258210 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:12.257842    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg" event={"ID":"3d2ca598-70b0-442d-a6d9-664ce9c28cca","Type":"ContainerStarted","Data":"b3e85491b4d29536a22f9347cb55284df08bc0688a5788841e0a64761deaa099"}
Nov 02 13:22:12.321460 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:12.321430    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:12.377334 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:12.377260    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg" podStartSLOduration=2.377243628 podStartE2EDuration="2.377243628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:22:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:22:12.373160471 +0000 UTC m=+5158.944510228" watchObservedRunningTime="2026-11-02 13:22:12.377243628 +0000 UTC m=+5158.948593386"
Nov 02 13:22:21.321264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:21.321222    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:21.333590 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:21.333567    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:21.368416 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:21.368391    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:22:30.982340 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:22:30.982252    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:22:47.679453 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:47.679393    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-7hsh2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:22:47.680268 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:22:47.680234    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:23:00.982358 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:23:00.982323    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:23:14.803408 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:14.803377    4942 scope.go:122] "RemoveContainer" containerID="8b0e114ad961d57335f743b66e1b4b745fb339a192faa0cd60b80e44945b1687"
Nov 02 13:23:20.420526 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:20.420495    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"]
Nov 02 13:23:30.982234 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:23:30.982200    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:23:35.439845 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.439784    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg" podUID="3d2ca598-70b0-442d-a6d9-664ce9c28cca" containerName="main" containerID="cri-o://198473588eb57f77d17d718957012cec6285b7b07c98b2566adc5a194d292d66" gracePeriod=45
Nov 02 13:23:35.689175 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.689146    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:23:35.691380 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.691337    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-tmp-dir\") pod \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") "
Nov 02 13:23:35.691452 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.691409    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-dshm\") pod \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") "
Nov 02 13:23:35.691571 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.691549    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-tmp-dir" pod "3d2ca598-70b0-442d-a6d9-664ce9c28cca" (UID: "3d2ca598-70b0-442d-a6d9-664ce9c28cca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:23:35.691727 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.691711    4942 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-tmp-dir\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:23:35.693410 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.693385    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-dshm" pod "3d2ca598-70b0-442d-a6d9-664ce9c28cca" (UID: "3d2ca598-70b0-442d-a6d9-664ce9c28cca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:23:35.792622 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.792596    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d2ca598-70b0-442d-a6d9-664ce9c28cca-kube-api-access-swmsh\" (UniqueName: \"kubernetes.io/projected/3d2ca598-70b0-442d-a6d9-664ce9c28cca-kube-api-access-swmsh\") pod \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") "
Nov 02 13:23:35.792745 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.792686    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-home\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-home\") pod \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") "
Nov 02 13:23:35.792745 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.792708    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3d2ca598-70b0-442d-a6d9-664ce9c28cca-tls-certs\" (UniqueName: \"kubernetes.io/secret/3d2ca598-70b0-442d-a6d9-664ce9c28cca-tls-certs\") pod \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") "
Nov 02 13:23:35.792745 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.792739    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-model-cache\") pod \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\" (UID: \"3d2ca598-70b0-442d-a6d9-664ce9c28cca\") "
Nov 02 13:23:35.793032 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.792982    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-home" pod "3d2ca598-70b0-442d-a6d9-664ce9c28cca" (UID: "3d2ca598-70b0-442d-a6d9-664ce9c28cca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:23:35.793095 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.793013    4942 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-dshm\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:23:35.793095 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.793057    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-model-cache" pod "3d2ca598-70b0-442d-a6d9-664ce9c28cca" (UID: "3d2ca598-70b0-442d-a6d9-664ce9c28cca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:23:35.794579 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.794560    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d2ca598-70b0-442d-a6d9-664ce9c28cca-kube-api-access-swmsh" pod "3d2ca598-70b0-442d-a6d9-664ce9c28cca" (UID: "3d2ca598-70b0-442d-a6d9-664ce9c28cca"). InnerVolumeSpecName "kube-api-access-swmsh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:23:35.794693 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.794623    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d2ca598-70b0-442d-a6d9-664ce9c28cca-tls-certs" pod "3d2ca598-70b0-442d-a6d9-664ce9c28cca" (UID: "3d2ca598-70b0-442d-a6d9-664ce9c28cca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:23:35.894056 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.894027    4942 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-home\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:23:35.894056 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.894054    4942 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d2ca598-70b0-442d-a6d9-664ce9c28cca-tls-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:23:35.894224 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.894063    4942 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d2ca598-70b0-442d-a6d9-664ce9c28cca-model-cache\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:23:35.894224 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.894073    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swmsh\" (UniqueName: \"kubernetes.io/projected/3d2ca598-70b0-442d-a6d9-664ce9c28cca-kube-api-access-swmsh\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:23:35.907715 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.907686    4942 generic.go:356] "Generic (PLEG): container finished" podID="3d2ca598-70b0-442d-a6d9-664ce9c28cca" containerID="198473588eb57f77d17d718957012cec6285b7b07c98b2566adc5a194d292d66" exitCode=0
Nov 02 13:23:35.907853 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.907725    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"
Nov 02 13:23:35.907853 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.907729    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg" event={"ID":"3d2ca598-70b0-442d-a6d9-664ce9c28cca","Type":"ContainerDied","Data":"198473588eb57f77d17d718957012cec6285b7b07c98b2566adc5a194d292d66"}
Nov 02 13:23:35.907853 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.907763    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg" event={"ID":"3d2ca598-70b0-442d-a6d9-664ce9c28cca","Type":"ContainerDied","Data":"b3e85491b4d29536a22f9347cb55284df08bc0688a5788841e0a64761deaa099"}
Nov 02 13:23:35.907853 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.907788    4942 scope.go:122] "RemoveContainer" containerID="198473588eb57f77d17d718957012cec6285b7b07c98b2566adc5a194d292d66"
Nov 02 13:23:35.917092 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.917076    4942 scope.go:122] "RemoveContainer" containerID="198473588eb57f77d17d718957012cec6285b7b07c98b2566adc5a194d292d66"
Nov 02 13:23:35.917386 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:23:35.917367    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"198473588eb57f77d17d718957012cec6285b7b07c98b2566adc5a194d292d66\": container with ID starting with 198473588eb57f77d17d718957012cec6285b7b07c98b2566adc5a194d292d66 not found: ID does not exist" containerID="198473588eb57f77d17d718957012cec6285b7b07c98b2566adc5a194d292d66"
Nov 02 13:23:35.917440 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:35.917395    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"198473588eb57f77d17d718957012cec6285b7b07c98b2566adc5a194d292d66"} err="failed to get container status \"198473588eb57f77d17d718957012cec6285b7b07c98b2566adc5a194d292d66\": rpc error: code = NotFound desc = could not find container \"198473588eb57f77d17d718957012cec6285b7b07c98b2566adc5a194d292d66\": container with ID starting with 198473588eb57f77d17d718957012cec6285b7b07c98b2566adc5a194d292d66 not found: ID does not exist"
Nov 02 13:23:36.073092 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:36.073024    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"]
Nov 02 13:23:36.079168 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:36.079141    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-128a019f-kserve-64d4bbd977-4blgg"]
Nov 02 13:23:37.541689 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:37.541658    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d2ca598-70b0-442d-a6d9-664ce9c28cca" path="/var/lib/kubelet/pods/3d2ca598-70b0-442d-a6d9-664ce9c28cca/volumes"
Nov 02 13:23:46.055580 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:46.055542    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs"]
Nov 02 13:23:46.114037 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:46.114004    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e49cfee7-pvtcs"]
Nov 02 13:23:47.559402 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:47.559358    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48" path="/var/lib/kubelet/pods/e3d787c6-07a7-4b8f-8ca6-7eeb7ab66e48/volumes"
Nov 02 13:23:47.680428 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:47.680390    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-7hsh2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:23:47.680785 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:23:47.680752    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:24:00.982517 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:24:00.982432    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:24:05.064531 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:05.064492    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9"]
Nov 02 13:24:05.110754 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:05.110705    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7d5a4698-gc9w9"]
Nov 02 13:24:05.545906 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:05.545865    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8726a12-ed2b-440d-9525-db2a39d769ec" path="/var/lib/kubelet/pods/c8726a12-ed2b-440d-9525-db2a39d769ec/volumes"
Nov 02 13:24:30.982643 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:24:30.982599    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:24:41.314412 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.314374    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"]
Nov 02 13:24:41.315051 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.314792    4942 cpu_manager.go:414] "containerMap: removing container" podUID="3d2ca598-70b0-442d-a6d9-664ce9c28cca" containerName="main"
Nov 02 13:24:41.315051 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.314804    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d2ca598-70b0-442d-a6d9-664ce9c28cca" containerName="main"
Nov 02 13:24:41.315051 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.314814    4942 cpu_manager.go:414] "containerMap: removing container" podUID="d0ffdc12-01f9-420e-95de-d980a47b4334" containerName="istio-proxy"
Nov 02 13:24:41.315051 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.314820    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0ffdc12-01f9-420e-95de-d980a47b4334" containerName="istio-proxy"
Nov 02 13:24:41.315051 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.314905    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d2ca598-70b0-442d-a6d9-664ce9c28cca" containerName="main"
Nov 02 13:24:41.318093 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.318069    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:24:41.370608 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.370576    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-1e6e34-serving-cert\""
Nov 02 13:24:41.370785 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.370752    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pl94d\""
Nov 02 13:24:41.419952 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.419923    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 02 13:24:41.422872 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.422850    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"]
Nov 02 13:24:41.457808 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.457774    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn\" (UID: \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:24:41.457982 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.457816    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wvwg\" (UniqueName: \"kubernetes.io/projected/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-kube-api-access-6wvwg\") pod \"maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn\" (UID: \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:24:41.457982 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.457898    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn\" (UID: \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:24:41.458090 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.458051    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-metrics-certs\") pod \"maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn\" (UID: \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:24:41.576902 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.576750    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-metrics-certs\") pod \"maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn\" (UID: \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:24:41.576902 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.576881    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn\" (UID: \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:24:41.577257 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.576911    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wvwg\" (UniqueName: \"kubernetes.io/projected/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-kube-api-access-6wvwg\") pod \"maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn\" (UID: \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:24:41.577257 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.576957    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn\" (UID: \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:24:41.577257 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:24:41.577190    4942 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-serving-cert: secret "maas-api-e2e-ipp-a-1e6e34-serving-cert" not found
Nov 02 13:24:41.577563 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:24:41.577275    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-maas-api-tls podName:373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9 nodeName:}" failed. No retries permitted until 2026-11-02 13:24:42.077252262 +0000 UTC m=+5308.648602000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-maas-api-tls") pod "maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn" (UID: "373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9") : secret "maas-api-e2e-ipp-a-1e6e34-serving-cert" not found
Nov 02 13:24:41.581897 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.581864    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-metrics-certs\") pod \"maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn\" (UID: \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:24:41.582504 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.582474    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn\" (UID: \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:24:41.642656 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:41.642620    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wvwg\" (UniqueName: \"kubernetes.io/projected/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-kube-api-access-6wvwg\") pod \"maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn\" (UID: \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:24:42.082249 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:42.082212    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn\" (UID: \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:24:42.084725 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:42.084697    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn\" (UID: \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:24:42.231370 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:42.231334    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:24:42.612052 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:42.611276    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"]
Nov 02 13:24:42.613014 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:24:42.612986    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod373b9f6e_8c2f_4a55_b3fb_65fdfcd930d9.slice/crio-d80b757fd2f07f98dc83ca3002a41a248ed94975acd73ce6df4e6f02e4a44eab WatchSource:0}: Error finding container d80b757fd2f07f98dc83ca3002a41a248ed94975acd73ce6df4e6f02e4a44eab: Status 404 returned error can't find the container with id d80b757fd2f07f98dc83ca3002a41a248ed94975acd73ce6df4e6f02e4a44eab
Nov 02 13:24:42.614406 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:42.614389    4942 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 13:24:43.435157 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:43.435111    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn" event={"ID":"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9","Type":"ContainerStarted","Data":"d80b757fd2f07f98dc83ca3002a41a248ed94975acd73ce6df4e6f02e4a44eab"}
Nov 02 13:24:44.441617 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:44.441585    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn" event={"ID":"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9","Type":"ContainerStarted","Data":"81d2e231e278a0cb2a398e6a82438b6459791bf42de38e2bf49d77f4692aa672"}
Nov 02 13:24:44.442040 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:44.441702    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:24:44.568562 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:44.568508    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn" podStartSLOduration=2.602846374 podStartE2EDuration="3.568494526s" totalImagesPullingTime="965.648152ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:24:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:24:44.567177444 +0000 UTC m=+5311.138527212" watchObservedRunningTime="2026-11-02 13:24:44.568494526 +0000 UTC m=+5311.139844310"
Nov 02 13:24:50.451474 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:24:50.451444    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:25:00.982686 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:25:00.982648    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:25:06.435814 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.435776    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"]
Nov 02 13:25:06.439519 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.439502    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.492139 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.492109    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-193883-openshift-default-dockercfg-ss75z\""
Nov 02 13:25:06.512774 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.512729    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-credential-socket\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.512966 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.512784    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-workload-socket\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.512966 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.512874    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxr64\" (UniqueName: \"kubernetes.io/projected/3abebf5d-924f-41d8-8c5d-afea677a88ff-kube-api-access-vxr64\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.512966 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.512942    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-podinfo\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.513157 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.512967    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-token\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.513157 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.512992    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-workload-certs\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.513157 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.513014    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-data\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.513157 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.513045    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-envoy\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.513157 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.513087    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3abebf5d-924f-41d8-8c5d-afea677a88ff-istiod-ca-cert\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.543595 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.543563    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"]
Nov 02 13:25:06.614392 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.614358    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-envoy\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.614613 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.614409    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3abebf5d-924f-41d8-8c5d-afea677a88ff-istiod-ca-cert\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.614613 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.614455    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-credential-socket\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.614613 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.614489    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-workload-socket\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.614613 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.614536    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxr64\" (UniqueName: \"kubernetes.io/projected/3abebf5d-924f-41d8-8c5d-afea677a88ff-kube-api-access-vxr64\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.614613 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.614603    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-podinfo\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.614852 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.614635    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-token\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.614912 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.614852    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-workload-certs\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.614912 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.614884    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-credential-socket\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.614912 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.614892    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-data\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.615219 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.615192    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-workload-socket\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.615342 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.615238    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-data\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.615342 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.615285    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3abebf5d-924f-41d8-8c5d-afea677a88ff-istiod-ca-cert\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.615453 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.615344    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-workload-certs\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.616808 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.616780    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-envoy\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.617182 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.617160    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-podinfo\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.675160 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.675122    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-token\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.675330 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.675289    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxr64\" (UniqueName: \"kubernetes.io/projected/3abebf5d-924f-41d8-8c5d-afea677a88ff-kube-api-access-vxr64\") pod \"e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") " pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:06.752619 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:06.752503    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:07.182852 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:07.181189    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"]
Nov 02 13:25:07.186310 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:25:07.186266    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3abebf5d_924f_41d8_8c5d_afea677a88ff.slice/crio-cfcfb300f80a0064f1bff1a4df400d41db0b8d86210b98ce2c544576e04c66de WatchSource:0}: Error finding container cfcfb300f80a0064f1bff1a4df400d41db0b8d86210b98ce2c544576e04c66de: Status 404 returned error can't find the container with id cfcfb300f80a0064f1bff1a4df400d41db0b8d86210b98ce2c544576e04c66de
Nov 02 13:25:07.188645 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:07.188609    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:25:07.188729 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:07.188679    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:25:07.188729 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:07.188705    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:25:07.575418 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:07.575327    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh" event={"ID":"3abebf5d-924f-41d8-8c5d-afea677a88ff","Type":"ContainerStarted","Data":"82004b96aca15662c7cb1caa045b4a8dafdff78f872743e07b2337e45b4652d7"}
Nov 02 13:25:07.575418 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:07.575379    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh" event={"ID":"3abebf5d-924f-41d8-8c5d-afea677a88ff","Type":"ContainerStarted","Data":"cfcfb300f80a0064f1bff1a4df400d41db0b8d86210b98ce2c544576e04c66de"}
Nov 02 13:25:07.696486 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:07.696414    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh" podStartSLOduration=1.6963965829999998 podStartE2EDuration="1.696396583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:25:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:25:07.69454567 +0000 UTC m=+5334.265895440" watchObservedRunningTime="2026-11-02 13:25:07.696396583 +0000 UTC m=+5334.267746342"
Nov 02 13:25:07.753321 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:07.753275    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:08.759616 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:08.759584    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:09.586627 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:09.586596    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:09.587648 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:09.587629    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:14.836568 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:14.836538    4942 scope.go:122] "RemoveContainer" containerID="316df60c3aef8a9672661038abf8a852f224e9047df466a41ff41a8bf3823d90"
Nov 02 13:25:14.845172 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:14.845150    4942 scope.go:122] "RemoveContainer" containerID="4d432c054c7a55aa8e4f7a57a84235e9c970acfb9757838758d25fe70542f05d"
Nov 02 13:25:24.613928 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:24.613852    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"]
Nov 02 13:25:24.618924 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:24.618904    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:25:24.672829 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:24.672792    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-193883-serving-cert\""
Nov 02 13:25:24.698336 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:24.698277    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/989735c9-b340-4730-8c00-c66515163d6c-metrics-certs\") pod \"maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g\" (UID: \"989735c9-b340-4730-8c00-c66515163d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:25:24.698336 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:24.698333    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/989735c9-b340-4730-8c00-c66515163d6c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g\" (UID: \"989735c9-b340-4730-8c00-c66515163d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:25:24.698557 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:24.698418    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7mpr\" (UniqueName: \"kubernetes.io/projected/989735c9-b340-4730-8c00-c66515163d6c-kube-api-access-j7mpr\") pod \"maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g\" (UID: \"989735c9-b340-4730-8c00-c66515163d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:25:24.698557 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:24.698449    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/989735c9-b340-4730-8c00-c66515163d6c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g\" (UID: \"989735c9-b340-4730-8c00-c66515163d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:25:24.724910 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:24.724882    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"]
Nov 02 13:25:24.799151 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:24.799118    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7mpr\" (UniqueName: \"kubernetes.io/projected/989735c9-b340-4730-8c00-c66515163d6c-kube-api-access-j7mpr\") pod \"maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g\" (UID: \"989735c9-b340-4730-8c00-c66515163d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:25:24.799151 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:24.799157    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/989735c9-b340-4730-8c00-c66515163d6c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g\" (UID: \"989735c9-b340-4730-8c00-c66515163d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:25:24.799430 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:24.799243    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/989735c9-b340-4730-8c00-c66515163d6c-metrics-certs\") pod \"maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g\" (UID: \"989735c9-b340-4730-8c00-c66515163d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:25:24.799430 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:24.799271    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/989735c9-b340-4730-8c00-c66515163d6c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g\" (UID: \"989735c9-b340-4730-8c00-c66515163d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:25:24.799430 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:25:24.799346    4942 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-serving-cert: secret "maas-api-e2e-ipp-b-193883-serving-cert" not found
Nov 02 13:25:24.799430 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:25:24.799412    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/989735c9-b340-4730-8c00-c66515163d6c-maas-api-tls podName:989735c9-b340-4730-8c00-c66515163d6c nodeName:}" failed. No retries permitted until 2026-11-02 13:25:25.299389411 +0000 UTC m=+5351.870739146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/989735c9-b340-4730-8c00-c66515163d6c-maas-api-tls") pod "maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g" (UID: "989735c9-b340-4730-8c00-c66515163d6c") : secret "maas-api-e2e-ipp-b-193883-serving-cert" not found
Nov 02 13:25:24.799930 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:24.799904    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/989735c9-b340-4730-8c00-c66515163d6c-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g\" (UID: \"989735c9-b340-4730-8c00-c66515163d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:25:24.801607 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:24.801590    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/989735c9-b340-4730-8c00-c66515163d6c-metrics-certs\") pod \"maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g\" (UID: \"989735c9-b340-4730-8c00-c66515163d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:25:24.871694 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:24.871571    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7mpr\" (UniqueName: \"kubernetes.io/projected/989735c9-b340-4730-8c00-c66515163d6c-kube-api-access-j7mpr\") pod \"maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g\" (UID: \"989735c9-b340-4730-8c00-c66515163d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:25:25.305094 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:25.305009    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/989735c9-b340-4730-8c00-c66515163d6c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g\" (UID: \"989735c9-b340-4730-8c00-c66515163d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:25:25.307440 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:25.307413    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/989735c9-b340-4730-8c00-c66515163d6c-maas-api-tls\") pod \"maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g\" (UID: \"989735c9-b340-4730-8c00-c66515163d6c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:25:25.531109 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:25.531080    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:25:25.910085 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:25.910057    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"]
Nov 02 13:25:25.911486 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:25:25.911452    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod989735c9_b340_4730_8c00_c66515163d6c.slice/crio-ce09b6748e07ed99ba3f870bbb2300da236b4a7c51ee73ad405b144200e56759 WatchSource:0}: Error finding container ce09b6748e07ed99ba3f870bbb2300da236b4a7c51ee73ad405b144200e56759: Status 404 returned error can't find the container with id ce09b6748e07ed99ba3f870bbb2300da236b4a7c51ee73ad405b144200e56759
Nov 02 13:25:26.677955 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:26.677919    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g" event={"ID":"989735c9-b340-4730-8c00-c66515163d6c","Type":"ContainerStarted","Data":"ce09b6748e07ed99ba3f870bbb2300da236b4a7c51ee73ad405b144200e56759"}
Nov 02 13:25:27.690524 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:27.690481    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g" event={"ID":"989735c9-b340-4730-8c00-c66515163d6c","Type":"ContainerStarted","Data":"e8dbd35ae9edbacd4fb9f2969e782b148868a79a3ca98ce609106aef7c16cec2"}
Nov 02 13:25:27.690971 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:27.690624    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:25:27.810646 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:27.810594    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g" podStartSLOduration=2.918848457 podStartE2EDuration="3.810581016s" totalImagesPullingTime="891.732559ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:25:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:25:27.808541464 +0000 UTC m=+5354.379891222" watchObservedRunningTime="2026-11-02 13:25:27.810581016 +0000 UTC m=+5354.381930802"
Nov 02 13:25:30.983813 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:25:30.983771    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:25:33.703518 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:33.703489    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:25:40.074726 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:40.074690    4942 patch_prober.go:29] interesting pod/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": read tcp 10.132.0.2:47164->10.132.0.133:15021: read: connection reset by peer" start-of-body=
Nov 02 13:25:40.075171 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:40.074766    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh" podUID="3abebf5d-924f-41d8-8c5d-afea677a88ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": read tcp 10.132.0.2:47164->10.132.0.133:15021: read: connection reset by peer"
Nov 02 13:25:40.769435 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:40.769400    4942 generic.go:356] "Generic (PLEG): container finished" podID="3abebf5d-924f-41d8-8c5d-afea677a88ff" containerID="82004b96aca15662c7cb1caa045b4a8dafdff78f872743e07b2337e45b4652d7" exitCode=0
Nov 02 13:25:40.769619 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:40.769468    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh" event={"ID":"3abebf5d-924f-41d8-8c5d-afea677a88ff","Type":"ContainerDied","Data":"82004b96aca15662c7cb1caa045b4a8dafdff78f872743e07b2337e45b4652d7"}
Nov 02 13:25:40.769827 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:40.769811    4942 scope.go:122] "RemoveContainer" containerID="82004b96aca15662c7cb1caa045b4a8dafdff78f872743e07b2337e45b4652d7"
Nov 02 13:25:40.771082 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:40.771054    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:25:40.771168 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:40.771122    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:25:40.771168 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:40.771154    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:25:41.784802 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:41.784760    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh" event={"ID":"3abebf5d-924f-41d8-8c5d-afea677a88ff","Type":"ContainerStarted","Data":"5e955360fb83d7dbbd8194d95106d73a6e43e96e34aaf7ea839913de96c2f015"}
Nov 02 13:25:42.773323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:42.771652    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:42.773323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:42.771782    4942 patch_prober.go:29] interesting pod/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Nov 02 13:25:42.773323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:42.771862    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh" podUID="3abebf5d-924f-41d8-8c5d-afea677a88ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Nov 02 13:25:43.760846 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:43.760803    4942 patch_prober.go:29] interesting pod/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Nov 02 13:25:43.761459 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:43.761429    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh" podUID="3abebf5d-924f-41d8-8c5d-afea677a88ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Nov 02 13:25:44.767722 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:44.767682    4942 patch_prober.go:29] interesting pod/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Nov 02 13:25:44.768350 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:44.768286    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh" podUID="3abebf5d-924f-41d8-8c5d-afea677a88ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Nov 02 13:25:45.763322 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:45.762689    4942 patch_prober.go:29] interesting pod/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Nov 02 13:25:45.763322 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:45.762765    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh" podUID="3abebf5d-924f-41d8-8c5d-afea677a88ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Nov 02 13:25:46.753572 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:46.753527    4942 patch_prober.go:29] interesting pod/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused" start-of-body=
Nov 02 13:25:46.754039 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:46.753600    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh" podUID="3abebf5d-924f-41d8-8c5d-afea677a88ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.133:15021/healthz/ready\": dial tcp 10.132.0.133:15021: connect: connection refused"
Nov 02 13:25:47.757353 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:47.757321    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:47.757795 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:47.757723    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:25:47.758442 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:25:47.758427    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:26:00.982814 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:26:00.982779    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:26:14.285798 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:26:14.285762    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:26:14.304852 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:26:14.304821    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:26:30.983744 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:26:30.983710    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:27:00.983758 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:27:00.983669    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:27:30.983626 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:27:30.983591    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:28:00.982874 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:28:00.982834    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:28:04.666711 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:04.666680    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"]
Nov 02 13:28:04.667277 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:04.666863    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn" podUID="373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9" containerName="maas-api" containerID="cri-o://81d2e231e278a0cb2a398e6a82438b6459791bf42de38e2bf49d77f4692aa672" gracePeriod=30
Nov 02 13:28:04.930285 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:04.930228    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:28:05.056700 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.056663    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-metrics-certs\") pod \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\" (UID: \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\") "
Nov 02 13:28:05.056962 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.056904    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-maas-api-tls\") pod \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\" (UID: \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\") "
Nov 02 13:28:05.056962 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.056953    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-kube-api-access-6wvwg\" (UniqueName: \"kubernetes.io/projected/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-kube-api-access-6wvwg\") pod \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\" (UID: \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\") "
Nov 02 13:28:05.057223 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.057009    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-openshift-service-ca\") pod \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\" (UID: \"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9\") "
Nov 02 13:28:05.059287 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.059240    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-openshift-service-ca" pod "373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9" (UID: "373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:28:05.060745 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.060714    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-metrics-certs" pod "373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9" (UID: "373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:28:05.062393 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.062352    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-maas-api-tls" pod "373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9" (UID: "373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:28:05.062609 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.062588    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-kube-api-access-6wvwg" pod "373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9" (UID: "373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9"). InnerVolumeSpecName "kube-api-access-6wvwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:28:05.159052 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.159015    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:28:05.159052 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.159046    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:28:05.159266 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.159060    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wvwg\" (UniqueName: \"kubernetes.io/projected/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-kube-api-access-6wvwg\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:28:05.159266 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.159071    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:28:05.572808 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.572776    4942 generic.go:356] "Generic (PLEG): container finished" podID="373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9" containerID="81d2e231e278a0cb2a398e6a82438b6459791bf42de38e2bf49d77f4692aa672" exitCode=0
Nov 02 13:28:05.572984 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.572822    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn" event={"ID":"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9","Type":"ContainerDied","Data":"81d2e231e278a0cb2a398e6a82438b6459791bf42de38e2bf49d77f4692aa672"}
Nov 02 13:28:05.572984 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.572849    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"
Nov 02 13:28:05.572984 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.572878    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn" event={"ID":"373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9","Type":"ContainerDied","Data":"d80b757fd2f07f98dc83ca3002a41a248ed94975acd73ce6df4e6f02e4a44eab"}
Nov 02 13:28:05.572984 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.572910    4942 scope.go:122] "RemoveContainer" containerID="81d2e231e278a0cb2a398e6a82438b6459791bf42de38e2bf49d77f4692aa672"
Nov 02 13:28:05.583502 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.583453    4942 scope.go:122] "RemoveContainer" containerID="81d2e231e278a0cb2a398e6a82438b6459791bf42de38e2bf49d77f4692aa672"
Nov 02 13:28:05.583799 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:28:05.583759    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81d2e231e278a0cb2a398e6a82438b6459791bf42de38e2bf49d77f4692aa672\": container with ID starting with 81d2e231e278a0cb2a398e6a82438b6459791bf42de38e2bf49d77f4692aa672 not found: ID does not exist" containerID="81d2e231e278a0cb2a398e6a82438b6459791bf42de38e2bf49d77f4692aa672"
Nov 02 13:28:05.583906 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.583802    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81d2e231e278a0cb2a398e6a82438b6459791bf42de38e2bf49d77f4692aa672"} err="failed to get container status \"81d2e231e278a0cb2a398e6a82438b6459791bf42de38e2bf49d77f4692aa672\": rpc error: code = NotFound desc = could not find container \"81d2e231e278a0cb2a398e6a82438b6459791bf42de38e2bf49d77f4692aa672\": container with ID starting with 81d2e231e278a0cb2a398e6a82438b6459791bf42de38e2bf49d77f4692aa672 not found: ID does not exist"
Nov 02 13:28:05.740379 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.740345    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"]
Nov 02 13:28:05.744809 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:05.744781    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1e6e34-5689577f8d-c8cmn"]
Nov 02 13:28:07.541523 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:07.541489    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9" path="/var/lib/kubelet/pods/373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9/volumes"
Nov 02 13:28:30.982155 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:28:30.982069    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:28:35.194307 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.194265    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"]
Nov 02 13:28:35.194760 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.194486    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g" podUID="989735c9-b340-4730-8c00-c66515163d6c" containerName="maas-api" containerID="cri-o://e8dbd35ae9edbacd4fb9f2969e782b148868a79a3ca98ce609106aef7c16cec2" gracePeriod=30
Nov 02 13:28:35.461787 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.461758    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:28:35.556049 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.556021    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/989735c9-b340-4730-8c00-c66515163d6c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/989735c9-b340-4730-8c00-c66515163d6c-openshift-service-ca\") pod \"989735c9-b340-4730-8c00-c66515163d6c\" (UID: \"989735c9-b340-4730-8c00-c66515163d6c\") "
Nov 02 13:28:35.556214 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.556109    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/989735c9-b340-4730-8c00-c66515163d6c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/989735c9-b340-4730-8c00-c66515163d6c-maas-api-tls\") pod \"989735c9-b340-4730-8c00-c66515163d6c\" (UID: \"989735c9-b340-4730-8c00-c66515163d6c\") "
Nov 02 13:28:35.556214 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.556166    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/989735c9-b340-4730-8c00-c66515163d6c-kube-api-access-j7mpr\" (UniqueName: \"kubernetes.io/projected/989735c9-b340-4730-8c00-c66515163d6c-kube-api-access-j7mpr\") pod \"989735c9-b340-4730-8c00-c66515163d6c\" (UID: \"989735c9-b340-4730-8c00-c66515163d6c\") "
Nov 02 13:28:35.556351 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.556228    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/989735c9-b340-4730-8c00-c66515163d6c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/989735c9-b340-4730-8c00-c66515163d6c-metrics-certs\") pod \"989735c9-b340-4730-8c00-c66515163d6c\" (UID: \"989735c9-b340-4730-8c00-c66515163d6c\") "
Nov 02 13:28:35.556455 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.556426    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/989735c9-b340-4730-8c00-c66515163d6c-openshift-service-ca" pod "989735c9-b340-4730-8c00-c66515163d6c" (UID: "989735c9-b340-4730-8c00-c66515163d6c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:28:35.556620 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.556591    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/989735c9-b340-4730-8c00-c66515163d6c-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:28:35.558210 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.558187    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/989735c9-b340-4730-8c00-c66515163d6c-metrics-certs" pod "989735c9-b340-4730-8c00-c66515163d6c" (UID: "989735c9-b340-4730-8c00-c66515163d6c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:28:35.558333 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.558244    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/989735c9-b340-4730-8c00-c66515163d6c-maas-api-tls" pod "989735c9-b340-4730-8c00-c66515163d6c" (UID: "989735c9-b340-4730-8c00-c66515163d6c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:28:35.558333 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.558251    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/989735c9-b340-4730-8c00-c66515163d6c-kube-api-access-j7mpr" pod "989735c9-b340-4730-8c00-c66515163d6c" (UID: "989735c9-b340-4730-8c00-c66515163d6c"). InnerVolumeSpecName "kube-api-access-j7mpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:28:35.658150 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.658119    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/989735c9-b340-4730-8c00-c66515163d6c-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:28:35.658150 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.658146    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7mpr\" (UniqueName: \"kubernetes.io/projected/989735c9-b340-4730-8c00-c66515163d6c-kube-api-access-j7mpr\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:28:35.658150 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.658158    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/989735c9-b340-4730-8c00-c66515163d6c-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:28:35.739565 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.739472    4942 generic.go:356] "Generic (PLEG): container finished" podID="989735c9-b340-4730-8c00-c66515163d6c" containerID="e8dbd35ae9edbacd4fb9f2969e782b148868a79a3ca98ce609106aef7c16cec2" exitCode=0
Nov 02 13:28:35.739565 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.739523    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"
Nov 02 13:28:35.739738 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.739555    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g" event={"ID":"989735c9-b340-4730-8c00-c66515163d6c","Type":"ContainerDied","Data":"e8dbd35ae9edbacd4fb9f2969e782b148868a79a3ca98ce609106aef7c16cec2"}
Nov 02 13:28:35.739738 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.739616    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g" event={"ID":"989735c9-b340-4730-8c00-c66515163d6c","Type":"ContainerDied","Data":"ce09b6748e07ed99ba3f870bbb2300da236b4a7c51ee73ad405b144200e56759"}
Nov 02 13:28:35.739738 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.739638    4942 scope.go:122] "RemoveContainer" containerID="e8dbd35ae9edbacd4fb9f2969e782b148868a79a3ca98ce609106aef7c16cec2"
Nov 02 13:28:35.749721 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.749702    4942 scope.go:122] "RemoveContainer" containerID="e8dbd35ae9edbacd4fb9f2969e782b148868a79a3ca98ce609106aef7c16cec2"
Nov 02 13:28:35.749994 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:28:35.749975    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8dbd35ae9edbacd4fb9f2969e782b148868a79a3ca98ce609106aef7c16cec2\": container with ID starting with e8dbd35ae9edbacd4fb9f2969e782b148868a79a3ca98ce609106aef7c16cec2 not found: ID does not exist" containerID="e8dbd35ae9edbacd4fb9f2969e782b148868a79a3ca98ce609106aef7c16cec2"
Nov 02 13:28:35.750058 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.750007    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8dbd35ae9edbacd4fb9f2969e782b148868a79a3ca98ce609106aef7c16cec2"} err="failed to get container status \"e8dbd35ae9edbacd4fb9f2969e782b148868a79a3ca98ce609106aef7c16cec2\": rpc error: code = NotFound desc = could not find container \"e8dbd35ae9edbacd4fb9f2969e782b148868a79a3ca98ce609106aef7c16cec2\": container with ID starting with e8dbd35ae9edbacd4fb9f2969e782b148868a79a3ca98ce609106aef7c16cec2 not found: ID does not exist"
Nov 02 13:28:35.905963 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.905928    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"]
Nov 02 13:28:35.909334 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:35.909309    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-193883-64cfbb9b7-9455g"]
Nov 02 13:28:37.542475 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:37.542442    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="989735c9-b340-4730-8c00-c66515163d6c" path="/var/lib/kubelet/pods/989735c9-b340-4730-8c00-c66515163d6c/volumes"
Nov 02 13:28:42.266954 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:42.266913    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"]
Nov 02 13:28:42.267452 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:42.267168    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh" podUID="3abebf5d-924f-41d8-8c5d-afea677a88ff" containerName="istio-proxy" containerID="cri-o://5e955360fb83d7dbbd8194d95106d73a6e43e96e34aaf7ea839913de96c2f015" gracePeriod=30
Nov 02 13:28:47.521560 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.521531    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:28:47.574895 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.574836    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-podinfo\") pod \"3abebf5d-924f-41d8-8c5d-afea677a88ff\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") "
Nov 02 13:28:47.574895 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.574883    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-token\" (UniqueName: \"kubernetes.io/projected/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-token\") pod \"3abebf5d-924f-41d8-8c5d-afea677a88ff\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") "
Nov 02 13:28:47.575079 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.574903    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-envoy\") pod \"3abebf5d-924f-41d8-8c5d-afea677a88ff\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") "
Nov 02 13:28:47.575079 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.575028    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-data\") pod \"3abebf5d-924f-41d8-8c5d-afea677a88ff\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") "
Nov 02 13:28:47.575197 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.575097    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3abebf5d-924f-41d8-8c5d-afea677a88ff-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3abebf5d-924f-41d8-8c5d-afea677a88ff-istiod-ca-cert\") pod \"3abebf5d-924f-41d8-8c5d-afea677a88ff\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") "
Nov 02 13:28:47.575197 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.575150    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3abebf5d-924f-41d8-8c5d-afea677a88ff-kube-api-access-vxr64\" (UniqueName: \"kubernetes.io/projected/3abebf5d-924f-41d8-8c5d-afea677a88ff-kube-api-access-vxr64\") pod \"3abebf5d-924f-41d8-8c5d-afea677a88ff\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") "
Nov 02 13:28:47.575323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.575196    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-credential-socket\") pod \"3abebf5d-924f-41d8-8c5d-afea677a88ff\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") "
Nov 02 13:28:47.575389 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.575323    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-data" pod "3abebf5d-924f-41d8-8c5d-afea677a88ff" (UID: "3abebf5d-924f-41d8-8c5d-afea677a88ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:28:47.575389 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.575336    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-workload-socket\") pod \"3abebf5d-924f-41d8-8c5d-afea677a88ff\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") "
Nov 02 13:28:47.575502 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.575408    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-workload-certs\") pod \"3abebf5d-924f-41d8-8c5d-afea677a88ff\" (UID: \"3abebf5d-924f-41d8-8c5d-afea677a88ff\") "
Nov 02 13:28:47.575570 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.575549    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-workload-socket" pod "3abebf5d-924f-41d8-8c5d-afea677a88ff" (UID: "3abebf5d-924f-41d8-8c5d-afea677a88ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:28:47.575626 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.575567    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-credential-socket" pod "3abebf5d-924f-41d8-8c5d-afea677a88ff" (UID: "3abebf5d-924f-41d8-8c5d-afea677a88ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:28:47.575811 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.575769    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-workload-certs" pod "3abebf5d-924f-41d8-8c5d-afea677a88ff" (UID: "3abebf5d-924f-41d8-8c5d-afea677a88ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:28:47.575927 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.575816    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3abebf5d-924f-41d8-8c5d-afea677a88ff-istiod-ca-cert" pod "3abebf5d-924f-41d8-8c5d-afea677a88ff" (UID: "3abebf5d-924f-41d8-8c5d-afea677a88ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:28:47.575927 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.575852    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:28:47.575927 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.575868    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:28:47.575927 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.575878    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:28:47.577088 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.577069    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-token" pod "3abebf5d-924f-41d8-8c5d-afea677a88ff" (UID: "3abebf5d-924f-41d8-8c5d-afea677a88ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:28:47.577269 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.577247    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-podinfo" pod "3abebf5d-924f-41d8-8c5d-afea677a88ff" (UID: "3abebf5d-924f-41d8-8c5d-afea677a88ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 13:28:47.577441 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.577423    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-envoy" pod "3abebf5d-924f-41d8-8c5d-afea677a88ff" (UID: "3abebf5d-924f-41d8-8c5d-afea677a88ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:28:47.577555 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.577533    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3abebf5d-924f-41d8-8c5d-afea677a88ff-kube-api-access-vxr64" pod "3abebf5d-924f-41d8-8c5d-afea677a88ff" (UID: "3abebf5d-924f-41d8-8c5d-afea677a88ff"). InnerVolumeSpecName "kube-api-access-vxr64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:28:47.676515 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.676481    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:28:47.676515 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.676514    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:28:47.676515 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.676524    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3abebf5d-924f-41d8-8c5d-afea677a88ff-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:28:47.676515 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.676533    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxr64\" (UniqueName: \"kubernetes.io/projected/3abebf5d-924f-41d8-8c5d-afea677a88ff-kube-api-access-vxr64\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:28:47.676768 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.676542    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3abebf5d-924f-41d8-8c5d-afea677a88ff-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:28:47.676768 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.676550    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3abebf5d-924f-41d8-8c5d-afea677a88ff-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:28:47.807005 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.806977    4942 generic.go:356] "Generic (PLEG): container finished" podID="3abebf5d-924f-41d8-8c5d-afea677a88ff" containerID="5e955360fb83d7dbbd8194d95106d73a6e43e96e34aaf7ea839913de96c2f015" exitCode=0
Nov 02 13:28:47.807161 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.807045    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"
Nov 02 13:28:47.807161 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.807054    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh" event={"ID":"3abebf5d-924f-41d8-8c5d-afea677a88ff","Type":"ContainerDied","Data":"5e955360fb83d7dbbd8194d95106d73a6e43e96e34aaf7ea839913de96c2f015"}
Nov 02 13:28:47.807161 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.807116    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh" event={"ID":"3abebf5d-924f-41d8-8c5d-afea677a88ff","Type":"ContainerDied","Data":"cfcfb300f80a0064f1bff1a4df400d41db0b8d86210b98ce2c544576e04c66de"}
Nov 02 13:28:47.807161 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.807149    4942 scope.go:122] "RemoveContainer" containerID="5e955360fb83d7dbbd8194d95106d73a6e43e96e34aaf7ea839913de96c2f015"
Nov 02 13:28:47.820231 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.820213    4942 scope.go:122] "RemoveContainer" containerID="82004b96aca15662c7cb1caa045b4a8dafdff78f872743e07b2337e45b4652d7"
Nov 02 13:28:47.828114 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.828094    4942 scope.go:122] "RemoveContainer" containerID="5e955360fb83d7dbbd8194d95106d73a6e43e96e34aaf7ea839913de96c2f015"
Nov 02 13:28:47.828397 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:28:47.828374    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e955360fb83d7dbbd8194d95106d73a6e43e96e34aaf7ea839913de96c2f015\": container with ID starting with 5e955360fb83d7dbbd8194d95106d73a6e43e96e34aaf7ea839913de96c2f015 not found: ID does not exist" containerID="5e955360fb83d7dbbd8194d95106d73a6e43e96e34aaf7ea839913de96c2f015"
Nov 02 13:28:47.828457 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.828408    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e955360fb83d7dbbd8194d95106d73a6e43e96e34aaf7ea839913de96c2f015"} err="failed to get container status \"5e955360fb83d7dbbd8194d95106d73a6e43e96e34aaf7ea839913de96c2f015\": rpc error: code = NotFound desc = could not find container \"5e955360fb83d7dbbd8194d95106d73a6e43e96e34aaf7ea839913de96c2f015\": container with ID starting with 5e955360fb83d7dbbd8194d95106d73a6e43e96e34aaf7ea839913de96c2f015 not found: ID does not exist"
Nov 02 13:28:47.828457 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.828426    4942 scope.go:122] "RemoveContainer" containerID="82004b96aca15662c7cb1caa045b4a8dafdff78f872743e07b2337e45b4652d7"
Nov 02 13:28:47.828668 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:28:47.828649    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82004b96aca15662c7cb1caa045b4a8dafdff78f872743e07b2337e45b4652d7\": container with ID starting with 82004b96aca15662c7cb1caa045b4a8dafdff78f872743e07b2337e45b4652d7 not found: ID does not exist" containerID="82004b96aca15662c7cb1caa045b4a8dafdff78f872743e07b2337e45b4652d7"
Nov 02 13:28:47.828726 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.828677    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82004b96aca15662c7cb1caa045b4a8dafdff78f872743e07b2337e45b4652d7"} err="failed to get container status \"82004b96aca15662c7cb1caa045b4a8dafdff78f872743e07b2337e45b4652d7\": rpc error: code = NotFound desc = could not find container \"82004b96aca15662c7cb1caa045b4a8dafdff78f872743e07b2337e45b4652d7\": container with ID starting with 82004b96aca15662c7cb1caa045b4a8dafdff78f872743e07b2337e45b4652d7 not found: ID does not exist"
Nov 02 13:28:47.971915 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.971886    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"]
Nov 02 13:28:47.977618 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:47.977588    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-193883-openshift-default-9d786df8f-9pffh"]
Nov 02 13:28:49.542314 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:28:49.542269    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3abebf5d-924f-41d8-8c5d-afea677a88ff" path="/var/lib/kubelet/pods/3abebf5d-924f-41d8-8c5d-afea677a88ff/volumes"
Nov 02 13:29:00.982591 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:29:00.982558    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:29:30.981646 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:29:30.981607    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:29:32.306285 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:32.306247    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"]
Nov 02 13:29:32.306712 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:32.306521    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8" podUID="cc69931c-3e6a-4a50-b706-5805a16167b0" containerName="istio-proxy" containerID="cri-o://4940097a7fac4f1e49271ba6f31935a1f3b8998c31ca7a374c1fbfdfee8d4fe6" gracePeriod=30
Nov 02 13:29:32.359373 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:32.359342    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"]
Nov 02 13:29:32.359550 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:32.359531    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt" podUID="eee44d9a-da91-4564-8b1e-efda1f126db6" containerName="istio-proxy" containerID="cri-o://4392ce405ba65678f2a09c51bf86304d0ed48033e3775d65efbb4bb9e76c90e6" gracePeriod=30
Nov 02 13:29:37.745103 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.745080    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:29:37.781327 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.781293    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:29:37.910332 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.910214    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc69931c-3e6a-4a50-b706-5805a16167b0-kube-api-access-9jcvs\" (UniqueName: \"kubernetes.io/projected/cc69931c-3e6a-4a50-b706-5805a16167b0-kube-api-access-9jcvs\") pod \"cc69931c-3e6a-4a50-b706-5805a16167b0\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") "
Nov 02 13:29:37.910332 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.910274    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-token\" (UniqueName: \"kubernetes.io/projected/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-token\") pod \"cc69931c-3e6a-4a50-b706-5805a16167b0\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") "
Nov 02 13:29:37.910586 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.910354    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-data\") pod \"cc69931c-3e6a-4a50-b706-5805a16167b0\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") "
Nov 02 13:29:37.910586 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.910389    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-token\" (UniqueName: \"kubernetes.io/projected/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-token\") pod \"eee44d9a-da91-4564-8b1e-efda1f126db6\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") "
Nov 02 13:29:37.910586 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.910447    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-envoy\") pod \"eee44d9a-da91-4564-8b1e-efda1f126db6\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") "
Nov 02 13:29:37.910586 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.910475    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-workload-certs\") pod \"cc69931c-3e6a-4a50-b706-5805a16167b0\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") "
Nov 02 13:29:37.910586 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.910519    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-workload-socket\") pod \"eee44d9a-da91-4564-8b1e-efda1f126db6\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") "
Nov 02 13:29:37.910586 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.910562    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-credential-socket\") pod \"eee44d9a-da91-4564-8b1e-efda1f126db6\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") "
Nov 02 13:29:37.910937 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.910597    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-podinfo\") pod \"cc69931c-3e6a-4a50-b706-5805a16167b0\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") "
Nov 02 13:29:37.910937 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.910626    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cc69931c-3e6a-4a50-b706-5805a16167b0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc69931c-3e6a-4a50-b706-5805a16167b0-istiod-ca-cert\") pod \"cc69931c-3e6a-4a50-b706-5805a16167b0\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") "
Nov 02 13:29:37.910937 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.910660    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-envoy\") pod \"cc69931c-3e6a-4a50-b706-5805a16167b0\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") "
Nov 02 13:29:37.910937 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.910717    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-credential-socket\") pod \"cc69931c-3e6a-4a50-b706-5805a16167b0\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") "
Nov 02 13:29:37.910937 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.910762    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-data\") pod \"eee44d9a-da91-4564-8b1e-efda1f126db6\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") "
Nov 02 13:29:37.910937 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.910807    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-workload-socket\") pod \"cc69931c-3e6a-4a50-b706-5805a16167b0\" (UID: \"cc69931c-3e6a-4a50-b706-5805a16167b0\") "
Nov 02 13:29:37.910937 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.910837    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-workload-certs\") pod \"eee44d9a-da91-4564-8b1e-efda1f126db6\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") "
Nov 02 13:29:37.910937 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.910882    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eee44d9a-da91-4564-8b1e-efda1f126db6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eee44d9a-da91-4564-8b1e-efda1f126db6-istiod-ca-cert\") pod \"eee44d9a-da91-4564-8b1e-efda1f126db6\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") "
Nov 02 13:29:37.910937 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.910911    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eee44d9a-da91-4564-8b1e-efda1f126db6-kube-api-access-gwjcb\" (UniqueName: \"kubernetes.io/projected/eee44d9a-da91-4564-8b1e-efda1f126db6-kube-api-access-gwjcb\") pod \"eee44d9a-da91-4564-8b1e-efda1f126db6\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") "
Nov 02 13:29:37.911474 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.910946    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-podinfo\") pod \"eee44d9a-da91-4564-8b1e-efda1f126db6\" (UID: \"eee44d9a-da91-4564-8b1e-efda1f126db6\") "
Nov 02 13:29:37.914405 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.911978    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-data" pod "eee44d9a-da91-4564-8b1e-efda1f126db6" (UID: "eee44d9a-da91-4564-8b1e-efda1f126db6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:29:37.914405 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.912227    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-credential-socket" pod "cc69931c-3e6a-4a50-b706-5805a16167b0" (UID: "cc69931c-3e6a-4a50-b706-5805a16167b0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:29:37.914405 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.913111    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-data" pod "cc69931c-3e6a-4a50-b706-5805a16167b0" (UID: "cc69931c-3e6a-4a50-b706-5805a16167b0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:29:37.914405 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.913412    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-workload-certs" pod "eee44d9a-da91-4564-8b1e-efda1f126db6" (UID: "eee44d9a-da91-4564-8b1e-efda1f126db6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:29:37.914405 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.913642    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eee44d9a-da91-4564-8b1e-efda1f126db6-istiod-ca-cert" pod "eee44d9a-da91-4564-8b1e-efda1f126db6" (UID: "eee44d9a-da91-4564-8b1e-efda1f126db6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:29:37.914405 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.913660    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc69931c-3e6a-4a50-b706-5805a16167b0-istiod-ca-cert" pod "cc69931c-3e6a-4a50-b706-5805a16167b0" (UID: "cc69931c-3e6a-4a50-b706-5805a16167b0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:29:37.914405 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.914325    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-workload-socket" pod "eee44d9a-da91-4564-8b1e-efda1f126db6" (UID: "eee44d9a-da91-4564-8b1e-efda1f126db6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:29:37.914938 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.914909    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-credential-socket" pod "eee44d9a-da91-4564-8b1e-efda1f126db6" (UID: "eee44d9a-da91-4564-8b1e-efda1f126db6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:29:37.915003 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.914983    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-workload-certs" pod "cc69931c-3e6a-4a50-b706-5805a16167b0" (UID: "cc69931c-3e6a-4a50-b706-5805a16167b0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:29:37.920610 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.920142    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-workload-socket" pod "cc69931c-3e6a-4a50-b706-5805a16167b0" (UID: "cc69931c-3e6a-4a50-b706-5805a16167b0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:29:37.922552 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.922512    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-token" pod "cc69931c-3e6a-4a50-b706-5805a16167b0" (UID: "cc69931c-3e6a-4a50-b706-5805a16167b0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:29:37.922769 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.922727    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eee44d9a-da91-4564-8b1e-efda1f126db6-kube-api-access-gwjcb" pod "eee44d9a-da91-4564-8b1e-efda1f126db6" (UID: "eee44d9a-da91-4564-8b1e-efda1f126db6"). InnerVolumeSpecName "kube-api-access-gwjcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:29:37.922950 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.922921    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-podinfo" pod "cc69931c-3e6a-4a50-b706-5805a16167b0" (UID: "cc69931c-3e6a-4a50-b706-5805a16167b0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 13:29:37.923130 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.923109    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-envoy" pod "eee44d9a-da91-4564-8b1e-efda1f126db6" (UID: "eee44d9a-da91-4564-8b1e-efda1f126db6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:29:37.923253 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.923125    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-podinfo" pod "eee44d9a-da91-4564-8b1e-efda1f126db6" (UID: "eee44d9a-da91-4564-8b1e-efda1f126db6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 13:29:37.923253 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.923238    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc69931c-3e6a-4a50-b706-5805a16167b0-kube-api-access-9jcvs" pod "cc69931c-3e6a-4a50-b706-5805a16167b0" (UID: "cc69931c-3e6a-4a50-b706-5805a16167b0"). InnerVolumeSpecName "kube-api-access-9jcvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:29:37.924093 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.924067    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-token" pod "eee44d9a-da91-4564-8b1e-efda1f126db6" (UID: "eee44d9a-da91-4564-8b1e-efda1f126db6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:29:37.924223 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:37.924131    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-envoy" pod "cc69931c-3e6a-4a50-b706-5805a16167b0" (UID: "cc69931c-3e6a-4a50-b706-5805a16167b0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:29:38.012264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.011894    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:29:38.012264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.011929    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:29:38.012264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.011946    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:29:38.012264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.011962    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:29:38.012264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.011977    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:29:38.012264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.011993    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:29:38.012264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.012007    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:29:38.012264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.012022    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc69931c-3e6a-4a50-b706-5805a16167b0-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:29:38.012264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.012036    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:29:38.012264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.012051    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:29:38.012264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.012064    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:29:38.012264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.012078    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc69931c-3e6a-4a50-b706-5805a16167b0-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:29:38.012264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.012092    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eee44d9a-da91-4564-8b1e-efda1f126db6-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:29:38.012264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.012105    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eee44d9a-da91-4564-8b1e-efda1f126db6-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:29:38.012264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.012120    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwjcb\" (UniqueName: \"kubernetes.io/projected/eee44d9a-da91-4564-8b1e-efda1f126db6-kube-api-access-gwjcb\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:29:38.012264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.012136    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eee44d9a-da91-4564-8b1e-efda1f126db6-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:29:38.012264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.012151    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jcvs\" (UniqueName: \"kubernetes.io/projected/cc69931c-3e6a-4a50-b706-5805a16167b0-kube-api-access-9jcvs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:29:38.013384 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.013351    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc69931c-3e6a-4a50-b706-5805a16167b0-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:29:38.128685 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.128480    4942 generic.go:356] "Generic (PLEG): container finished" podID="cc69931c-3e6a-4a50-b706-5805a16167b0" containerID="4940097a7fac4f1e49271ba6f31935a1f3b8998c31ca7a374c1fbfdfee8d4fe6" exitCode=0
Nov 02 13:29:38.128685 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.128637    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8" event={"ID":"cc69931c-3e6a-4a50-b706-5805a16167b0","Type":"ContainerDied","Data":"4940097a7fac4f1e49271ba6f31935a1f3b8998c31ca7a374c1fbfdfee8d4fe6"}
Nov 02 13:29:38.128685 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.128690    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8" event={"ID":"cc69931c-3e6a-4a50-b706-5805a16167b0","Type":"ContainerDied","Data":"f781fbafa5b34be8b22c0b16f099549fa92e58e2e80204917cdd89c912000674"}
Nov 02 13:29:38.129020 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.128721    4942 scope.go:122] "RemoveContainer" containerID="4940097a7fac4f1e49271ba6f31935a1f3b8998c31ca7a374c1fbfdfee8d4fe6"
Nov 02 13:29:38.129020 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.128877    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"
Nov 02 13:29:38.133261 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.133229    4942 generic.go:356] "Generic (PLEG): container finished" podID="eee44d9a-da91-4564-8b1e-efda1f126db6" containerID="4392ce405ba65678f2a09c51bf86304d0ed48033e3775d65efbb4bb9e76c90e6" exitCode=0
Nov 02 13:29:38.133405 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.133358    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt" event={"ID":"eee44d9a-da91-4564-8b1e-efda1f126db6","Type":"ContainerDied","Data":"4392ce405ba65678f2a09c51bf86304d0ed48033e3775d65efbb4bb9e76c90e6"}
Nov 02 13:29:38.133405 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.133400    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt" event={"ID":"eee44d9a-da91-4564-8b1e-efda1f126db6","Type":"ContainerDied","Data":"49f3bd7aa0189e991142575bd0575f74285dd9206f9ee7dd14ab0d5061eca7ea"}
Nov 02 13:29:38.133484 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.133468    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"
Nov 02 13:29:38.143260 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.143233    4942 scope.go:122] "RemoveContainer" containerID="4940097a7fac4f1e49271ba6f31935a1f3b8998c31ca7a374c1fbfdfee8d4fe6"
Nov 02 13:29:38.143628 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:29:38.143604    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4940097a7fac4f1e49271ba6f31935a1f3b8998c31ca7a374c1fbfdfee8d4fe6\": container with ID starting with 4940097a7fac4f1e49271ba6f31935a1f3b8998c31ca7a374c1fbfdfee8d4fe6 not found: ID does not exist" containerID="4940097a7fac4f1e49271ba6f31935a1f3b8998c31ca7a374c1fbfdfee8d4fe6"
Nov 02 13:29:38.143768 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.143741    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4940097a7fac4f1e49271ba6f31935a1f3b8998c31ca7a374c1fbfdfee8d4fe6"} err="failed to get container status \"4940097a7fac4f1e49271ba6f31935a1f3b8998c31ca7a374c1fbfdfee8d4fe6\": rpc error: code = NotFound desc = could not find container \"4940097a7fac4f1e49271ba6f31935a1f3b8998c31ca7a374c1fbfdfee8d4fe6\": container with ID starting with 4940097a7fac4f1e49271ba6f31935a1f3b8998c31ca7a374c1fbfdfee8d4fe6 not found: ID does not exist"
Nov 02 13:29:38.143856 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.143770    4942 scope.go:122] "RemoveContainer" containerID="4392ce405ba65678f2a09c51bf86304d0ed48033e3775d65efbb4bb9e76c90e6"
Nov 02 13:29:38.153593 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.153569    4942 scope.go:122] "RemoveContainer" containerID="4392ce405ba65678f2a09c51bf86304d0ed48033e3775d65efbb4bb9e76c90e6"
Nov 02 13:29:38.154101 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:29:38.154073    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4392ce405ba65678f2a09c51bf86304d0ed48033e3775d65efbb4bb9e76c90e6\": container with ID starting with 4392ce405ba65678f2a09c51bf86304d0ed48033e3775d65efbb4bb9e76c90e6 not found: ID does not exist" containerID="4392ce405ba65678f2a09c51bf86304d0ed48033e3775d65efbb4bb9e76c90e6"
Nov 02 13:29:38.154246 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.154220    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4392ce405ba65678f2a09c51bf86304d0ed48033e3775d65efbb4bb9e76c90e6"} err="failed to get container status \"4392ce405ba65678f2a09c51bf86304d0ed48033e3775d65efbb4bb9e76c90e6\": rpc error: code = NotFound desc = could not find container \"4392ce405ba65678f2a09c51bf86304d0ed48033e3775d65efbb4bb9e76c90e6\": container with ID starting with 4392ce405ba65678f2a09c51bf86304d0ed48033e3775d65efbb4bb9e76c90e6 not found: ID does not exist"
Nov 02 13:29:38.506113 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.506078    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"]
Nov 02 13:29:38.605188 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.605152    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-8gqnt"]
Nov 02 13:29:38.712459 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.712390    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"]
Nov 02 13:29:38.811923 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:38.811890    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-r8lg8"]
Nov 02 13:29:39.541797 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:39.541761    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc69931c-3e6a-4a50-b706-5805a16167b0" path="/var/lib/kubelet/pods/cc69931c-3e6a-4a50-b706-5805a16167b0/volumes"
Nov 02 13:29:39.542200 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:29:39.542186    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eee44d9a-da91-4564-8b1e-efda1f126db6" path="/var/lib/kubelet/pods/eee44d9a-da91-4564-8b1e-efda1f126db6/volumes"
Nov 02 13:30:00.982264 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:30:00.982178    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:30:02.293486 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.293451    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"]
Nov 02 13:30:02.293869 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.293851    4942 cpu_manager.go:414] "containerMap: removing container" podUID="eee44d9a-da91-4564-8b1e-efda1f126db6" containerName="istio-proxy"
Nov 02 13:30:02.293869 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.293863    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eee44d9a-da91-4564-8b1e-efda1f126db6" containerName="istio-proxy"
Nov 02 13:30:02.293965 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.293877    4942 cpu_manager.go:414] "containerMap: removing container" podUID="3abebf5d-924f-41d8-8c5d-afea677a88ff" containerName="istio-proxy"
Nov 02 13:30:02.293965 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.293883    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3abebf5d-924f-41d8-8c5d-afea677a88ff" containerName="istio-proxy"
Nov 02 13:30:02.293965 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.293893    4942 cpu_manager.go:414] "containerMap: removing container" podUID="989735c9-b340-4730-8c00-c66515163d6c" containerName="maas-api"
Nov 02 13:30:02.293965 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.293898    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="989735c9-b340-4730-8c00-c66515163d6c" containerName="maas-api"
Nov 02 13:30:02.293965 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.293912    4942 cpu_manager.go:414] "containerMap: removing container" podUID="cc69931c-3e6a-4a50-b706-5805a16167b0" containerName="istio-proxy"
Nov 02 13:30:02.293965 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.293917    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc69931c-3e6a-4a50-b706-5805a16167b0" containerName="istio-proxy"
Nov 02 13:30:02.293965 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.293936    4942 cpu_manager.go:414] "containerMap: removing container" podUID="373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9" containerName="maas-api"
Nov 02 13:30:02.293965 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.293941    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9" containerName="maas-api"
Nov 02 13:30:02.294238 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.293995    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="3abebf5d-924f-41d8-8c5d-afea677a88ff" containerName="istio-proxy"
Nov 02 13:30:02.294238 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.294004    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="373b9f6e-8c2f-4a55-b3fb-65fdfcd930d9" containerName="maas-api"
Nov 02 13:30:02.294238 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.294010    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="eee44d9a-da91-4564-8b1e-efda1f126db6" containerName="istio-proxy"
Nov 02 13:30:02.294238 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.294017    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="989735c9-b340-4730-8c00-c66515163d6c" containerName="maas-api"
Nov 02 13:30:02.294238 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.294024    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc69931c-3e6a-4a50-b706-5805a16167b0" containerName="istio-proxy"
Nov 02 13:30:02.297032 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.297015    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.323681 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.323657    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.323798 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.323693    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.323798 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.323720    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.323870 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.323808    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.323870 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.323861    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.323940 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.323890    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl6cp\" (UniqueName: \"kubernetes.io/projected/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-kube-api-access-xl6cp\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.323974 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.323930    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.324049 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.324029    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.324141 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.324128    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.420055 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.420023    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"]
Nov 02 13:30:02.424997 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.424971    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl6cp\" (UniqueName: \"kubernetes.io/projected/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-kube-api-access-xl6cp\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.425148 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.425002    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.425148 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.425021    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.425148 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.425048    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.425148 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.425080    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.425148 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.425100    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.425148 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.425120    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.425496 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.425162    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.425496 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.425239    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.425630 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.425478    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.425688 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.425630    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.425744 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.425723    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.425843 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.425821    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.425947 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.425925    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.427463 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.427430    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.427854 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.427834    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.518423 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.518389    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.518559 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.518507    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl6cp\" (UniqueName: \"kubernetes.io/projected/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-kube-api-access-xl6cp\") pod \"maas-default-gateway-openshift-default-6dcf757899-pjqbs\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:02.610769 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:02.610743    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:03.045901 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:03.045871    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"]
Nov 02 13:30:03.047951 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:30:03.047923    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1c1a136_bb43_4e4d_b47b_3d7c2af690ca.slice/crio-2a11d2d0bb418bb36e7b87a594b6994b006ebcac89b5eddf4d6d189a55df748b WatchSource:0}: Error finding container 2a11d2d0bb418bb36e7b87a594b6994b006ebcac89b5eddf4d6d189a55df748b: Status 404 returned error can't find the container with id 2a11d2d0bb418bb36e7b87a594b6994b006ebcac89b5eddf4d6d189a55df748b
Nov 02 13:30:03.050082 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:03.050049    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:30:03.050140 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:03.050116    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:30:03.050178 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:03.050145    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:30:03.284285 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:03.284243    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs" event={"ID":"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca","Type":"ContainerStarted","Data":"cbd95db86fefff5631431a68c4a1902eaac0b4a6cf4f5bca717f3cfabefe157a"}
Nov 02 13:30:03.284470 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:03.284324    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs" event={"ID":"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca","Type":"ContainerStarted","Data":"2a11d2d0bb418bb36e7b87a594b6994b006ebcac89b5eddf4d6d189a55df748b"}
Nov 02 13:30:03.506233 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:03.506168    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs" podStartSLOduration=1.5058710419999999 podStartE2EDuration="1.505871042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:30:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:30:03.454004465 +0000 UTC m=+5630.025354223" watchObservedRunningTime="2026-11-02 13:30:03.505871042 +0000 UTC m=+5630.077220801"
Nov 02 13:30:03.611895 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:03.611857    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:04.612820 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:04.612782    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-pjqbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Nov 02 13:30:04.613281 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:04.612844    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs" podUID="d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Nov 02 13:30:05.611956 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:05.611913    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-pjqbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Nov 02 13:30:05.612146 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:05.611977    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs" podUID="d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Nov 02 13:30:06.612113 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:06.612072    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-pjqbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Nov 02 13:30:06.612607 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:06.612131    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs" podUID="d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Nov 02 13:30:07.611566 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:07.611512    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-pjqbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Nov 02 13:30:07.611757 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:07.611607    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs" podUID="d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Nov 02 13:30:08.611389 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:08.611345    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-pjqbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Nov 02 13:30:08.611945 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:08.611420    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs" podUID="d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Nov 02 13:30:09.612058 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:09.612016    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-pjqbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Nov 02 13:30:09.612483 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:09.612082    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs" podUID="d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Nov 02 13:30:10.611828 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:10.611787    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-pjqbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Nov 02 13:30:10.612010 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:10.611865    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs" podUID="d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Nov 02 13:30:12.369471 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:12.369438    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:12.369937 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:12.369775    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:12.370650 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:12.370626    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:30:17.336779 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.336745    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"]
Nov 02 13:30:17.337179 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.337164    4942 cpu_manager.go:414] "containerMap: removing container" podUID="3abebf5d-924f-41d8-8c5d-afea677a88ff" containerName="istio-proxy"
Nov 02 13:30:17.337223 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.337180    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3abebf5d-924f-41d8-8c5d-afea677a88ff" containerName="istio-proxy"
Nov 02 13:30:17.337259 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.337253    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="3abebf5d-924f-41d8-8c5d-afea677a88ff" containerName="istio-proxy"
Nov 02 13:30:17.342194 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.342173    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.446761 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.446731    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"]
Nov 02 13:30:17.490022 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.489993    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d43b8d98-489b-4c47-8718-9d47509709b5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.490159 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.490028    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.490159 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.490063    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.490233 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.490156    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d43b8d98-489b-4c47-8718-9d47509709b5-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.490233 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.490213    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ljjw\" (UniqueName: \"kubernetes.io/projected/d43b8d98-489b-4c47-8718-9d47509709b5-kube-api-access-8ljjw\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.490456 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.490256    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d43b8d98-489b-4c47-8718-9d47509709b5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.490761 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.490733    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.490916 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.490895    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.490982 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.490931    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.592389 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.592322    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.592389 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.592360    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.592389 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.592386    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d43b8d98-489b-4c47-8718-9d47509709b5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.592676 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.592402    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.592676 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.592428    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.592676 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.592450    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d43b8d98-489b-4c47-8718-9d47509709b5-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.592676 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.592472    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ljjw\" (UniqueName: \"kubernetes.io/projected/d43b8d98-489b-4c47-8718-9d47509709b5-kube-api-access-8ljjw\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.592676 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.592521    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d43b8d98-489b-4c47-8718-9d47509709b5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.592676 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.592556    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.592967 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.592779    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.593006 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.592980    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.593046 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.593008    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.593209 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.593188    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.593431 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.593414    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d43b8d98-489b-4c47-8718-9d47509709b5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.594585 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.594567    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.595281 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.595263    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d43b8d98-489b-4c47-8718-9d47509709b5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.653291 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.653256    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d43b8d98-489b-4c47-8718-9d47509709b5-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.653603 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.653579    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ljjw\" (UniqueName: \"kubernetes.io/projected/d43b8d98-489b-4c47-8718-9d47509709b5-kube-api-access-8ljjw\") pod \"maas-default-gateway-openshift-default-6dcf757899-qzbzn\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:17.655221 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:17.655200    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:18.087001 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:30:18.086968    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd43b8d98_489b_4c47_8718_9d47509709b5.slice/crio-4eac5edc61f5a98c653baf372415aa8ca4aab3d32381526501bb9c4703f3a028 WatchSource:0}: Error finding container 4eac5edc61f5a98c653baf372415aa8ca4aab3d32381526501bb9c4703f3a028: Status 404 returned error can't find the container with id 4eac5edc61f5a98c653baf372415aa8ca4aab3d32381526501bb9c4703f3a028
Nov 02 13:30:18.089274 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:18.089242    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:30:18.089366 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:18.089350    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:30:18.089409 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:18.089385    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:30:18.089777 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:18.089754    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"]
Nov 02 13:30:18.368540 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:18.368489    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn" event={"ID":"d43b8d98-489b-4c47-8718-9d47509709b5","Type":"ContainerStarted","Data":"ad98961077edd5d62663d02aedda2cf80ea5c9da68de1d2709c4241b18a8a5da"}
Nov 02 13:30:18.368930 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:18.368562    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn" event={"ID":"d43b8d98-489b-4c47-8718-9d47509709b5","Type":"ContainerStarted","Data":"4eac5edc61f5a98c653baf372415aa8ca4aab3d32381526501bb9c4703f3a028"}
Nov 02 13:30:18.539281 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:18.539205    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn" podStartSLOduration=1.53918552 podStartE2EDuration="1.53918552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:30:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:30:18.538193703 +0000 UTC m=+5645.109543459" watchObservedRunningTime="2026-11-02 13:30:18.53918552 +0000 UTC m=+5645.110535385"
Nov 02 13:30:18.670406 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:18.663387    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:19.671322 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:19.663222    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qzbzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Nov 02 13:30:19.671322 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:19.663285    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn" podUID="d43b8d98-489b-4c47-8718-9d47509709b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Nov 02 13:30:20.671323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:20.656610    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qzbzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Nov 02 13:30:20.671323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:20.656666    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn" podUID="d43b8d98-489b-4c47-8718-9d47509709b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Nov 02 13:30:21.677667 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:21.677633    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qzbzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Nov 02 13:30:21.678090 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:21.677682    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn" podUID="d43b8d98-489b-4c47-8718-9d47509709b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Nov 02 13:30:22.664321 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:22.656611    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qzbzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Nov 02 13:30:22.664321 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:22.656673    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn" podUID="d43b8d98-489b-4c47-8718-9d47509709b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Nov 02 13:30:23.673322 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:23.672404    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qzbzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Nov 02 13:30:23.673322 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:23.672467    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn" podUID="d43b8d98-489b-4c47-8718-9d47509709b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Nov 02 13:30:24.656369 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:24.656334    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qzbzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Nov 02 13:30:24.656537 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:24.656386    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn" podUID="d43b8d98-489b-4c47-8718-9d47509709b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Nov 02 13:30:25.655543 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:25.655505    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qzbzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Nov 02 13:30:25.656011 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:25.655561    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn" podUID="d43b8d98-489b-4c47-8718-9d47509709b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Nov 02 13:30:27.395448 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:27.395417    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:27.395929 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:27.395846    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:27.396738 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:30:27.396716    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:30:30.983034 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:30:30.982992    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:31:00.982791 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:31:00.982752    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:31:14.321615 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:31:14.321588    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:31:14.342415 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:31:14.342389    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:31:28.419927 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:31:28.419440    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-pjqbs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:31:28.419927 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:31:28.419521    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs" podUID="d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:31:28.438461 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:31:28.438424    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qzbzn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:31:28.438597 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:31:28.438512    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn" podUID="d43b8d98-489b-4c47-8718-9d47509709b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:31:30.981813 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:31:30.981778    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:32:01.129709 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:32:01.129636    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:32:30.982225 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:32:30.982183    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:32:40.876471 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:32:40.876414    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wkxz9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:32:40.877261 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:32:40.877229    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:33:00.982395 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:33:00.982309    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:33:31.002248 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:33:31.002168    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:34:01.121594 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:34:01.121538    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:34:02.686346 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:34:02.677821    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-7hsh2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:34:02.686346 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:34:02.677899    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:34:25.882532 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:34:25.882422    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wkxz9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:34:25.883280 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:34:25.883245    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:34:30.982742 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:34:30.982694    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:35:00.982823 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:35:00.982784    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:35:17.689460 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:35:17.689416    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-7hsh2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:35:17.690237 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:35:17.690200    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:35:30.984376 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:35:30.984326    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:35:58.389521 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:35:58.389417    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-pjqbs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:35:58.395630 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:35:58.390170    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs" podUID="d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:36:00.984025 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:36:00.983989    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:36:14.357696 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:36:14.357664    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:36:14.379316 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:36:14.379267    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:36:30.983545 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:36:30.983507    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:37:00.983444 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:37:00.983398    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:37:01.837581 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:01.837543    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh"]
Nov 02 13:37:01.854862 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:01.854833    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh"
Nov 02 13:37:01.903741 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:01.903705    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh"]
Nov 02 13:37:01.950041 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:01.950008    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-7ddf45\""
Nov 02 13:37:01.950221 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:01.950129    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-62e22d3b-dockercfg-cqwr7\""
Nov 02 13:37:02.002487 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:02.002450    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lv7g\" (UniqueName: \"kubernetes.io/projected/3c454dd8-d004-41c4-a978-c24ff8925ab7-kube-api-access-5lv7g\") pod \"payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh\" (UID: \"3c454dd8-d004-41c4-a978-c24ff8925ab7\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh"
Nov 02 13:37:02.002874 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:02.002497    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c454dd8-d004-41c4-a978-c24ff8925ab7-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh\" (UID: \"3c454dd8-d004-41c4-a978-c24ff8925ab7\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh"
Nov 02 13:37:02.103189 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:02.103159    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lv7g\" (UniqueName: \"kubernetes.io/projected/3c454dd8-d004-41c4-a978-c24ff8925ab7-kube-api-access-5lv7g\") pod \"payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh\" (UID: \"3c454dd8-d004-41c4-a978-c24ff8925ab7\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh"
Nov 02 13:37:02.103365 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:02.103203    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c454dd8-d004-41c4-a978-c24ff8925ab7-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh\" (UID: \"3c454dd8-d004-41c4-a978-c24ff8925ab7\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh"
Nov 02 13:37:02.104025 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:02.104003    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c454dd8-d004-41c4-a978-c24ff8925ab7-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh\" (UID: \"3c454dd8-d004-41c4-a978-c24ff8925ab7\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh"
Nov 02 13:37:02.163926 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:02.163888    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lv7g\" (UniqueName: \"kubernetes.io/projected/3c454dd8-d004-41c4-a978-c24ff8925ab7-kube-api-access-5lv7g\") pod \"payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh\" (UID: \"3c454dd8-d004-41c4-a978-c24ff8925ab7\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh"
Nov 02 13:37:02.169717 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:02.169691    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh"
Nov 02 13:37:02.545254 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:02.545219    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh"]
Nov 02 13:37:02.547438 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:37:02.547402    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c454dd8_d004_41c4_a978_c24ff8925ab7.slice/crio-514e363961ceca48da7fd687720b6b10ecb11d5f10b0f0885a3b278432c11c29 WatchSource:0}: Error finding container 514e363961ceca48da7fd687720b6b10ecb11d5f10b0f0885a3b278432c11c29: Status 404 returned error can't find the container with id 514e363961ceca48da7fd687720b6b10ecb11d5f10b0f0885a3b278432c11c29
Nov 02 13:37:03.275316 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:03.275263    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh" event={"ID":"3c454dd8-d004-41c4-a978-c24ff8925ab7","Type":"ContainerStarted","Data":"940fd10b7ab17c3aee50f05754ae073da995ac72e07c49a45409a60f9ed1e126"}
Nov 02 13:37:03.275780 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:03.275369    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh"
Nov 02 13:37:03.275780 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:03.275388    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh" event={"ID":"3c454dd8-d004-41c4-a978-c24ff8925ab7","Type":"ContainerStarted","Data":"514e363961ceca48da7fd687720b6b10ecb11d5f10b0f0885a3b278432c11c29"}
Nov 02 13:37:03.391764 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:03.391704    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh" podStartSLOduration=2.391690994 podStartE2EDuration="2.391690994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:37:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:37:03.390113806 +0000 UTC m=+6049.961463562" watchObservedRunningTime="2026-11-02 13:37:03.391690994 +0000 UTC m=+6049.963040774"
Nov 02 13:37:04.337638 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:04.337600    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"]
Nov 02 13:37:04.341133 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:04.341108    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:37:04.393316 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:04.393269    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-7ddf45-serving-cert\""
Nov 02 13:37:04.393525 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:04.393496    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 02 13:37:04.393590 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:04.393552    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pl94d\""
Nov 02 13:37:04.425968 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:04.425934    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k\" (UID: \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:37:04.426174 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:04.426149    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twfvn\" (UniqueName: \"kubernetes.io/projected/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-kube-api-access-twfvn\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k\" (UID: \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:37:04.426237 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:04.426184    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k\" (UID: \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:37:04.426237 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:04.426207    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-metrics-certs\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k\" (UID: \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:37:04.445429 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:04.445393    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"]
Nov 02 13:37:04.527104 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:04.527062    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k\" (UID: \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:37:04.527286 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:04.527155    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twfvn\" (UniqueName: \"kubernetes.io/projected/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-kube-api-access-twfvn\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k\" (UID: \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:37:04.527286 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:04.527177    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k\" (UID: \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:37:04.527286 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:04.527196    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-metrics-certs\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k\" (UID: \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:37:04.527511 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:37:04.527311    4942 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-serving-cert: secret "maas-api-e2e-worker-single-main-7ddf45-serving-cert" not found
Nov 02 13:37:04.527511 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:37:04.527378    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-maas-api-tls podName:d8b7911a-aa96-4cf8-a47d-67fa2160dfc8 nodeName:}" failed. No retries permitted until 2026-11-02 13:37:05.027363036 +0000 UTC m=+6051.598712771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-maas-api-tls") pod "maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k" (UID: "d8b7911a-aa96-4cf8-a47d-67fa2160dfc8") : secret "maas-api-e2e-worker-single-main-7ddf45-serving-cert" not found
Nov 02 13:37:04.527900 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:04.527874    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k\" (UID: \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:37:04.529652 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:04.529635    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-metrics-certs\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k\" (UID: \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:37:04.588879 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:04.588779    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twfvn\" (UniqueName: \"kubernetes.io/projected/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-kube-api-access-twfvn\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k\" (UID: \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:37:05.031565 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:05.031525    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k\" (UID: \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:37:05.034177 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:05.034138    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k\" (UID: \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:37:05.254757 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:05.254719    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:37:05.632081 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:05.631874    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"]
Nov 02 13:37:05.634582 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:37:05.634543    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8b7911a_aa96_4cf8_a47d_67fa2160dfc8.slice/crio-2a714363b371c595a21e5c801248251422497cfb0b62a38656b2c346313e3eba WatchSource:0}: Error finding container 2a714363b371c595a21e5c801248251422497cfb0b62a38656b2c346313e3eba: Status 404 returned error can't find the container with id 2a714363b371c595a21e5c801248251422497cfb0b62a38656b2c346313e3eba
Nov 02 13:37:05.636133 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:05.636096    4942 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 13:37:06.294829 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:06.294790    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k" event={"ID":"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8","Type":"ContainerStarted","Data":"2a714363b371c595a21e5c801248251422497cfb0b62a38656b2c346313e3eba"}
Nov 02 13:37:07.302017 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:07.301981    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k" event={"ID":"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8","Type":"ContainerStarted","Data":"c57ffa9a80df79baa7f1908dfa60bd7da024829440334f5af37e6f28e3be2a97"}
Nov 02 13:37:07.302479 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:07.302081    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:37:07.422179 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:07.422118    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k" podStartSLOduration=2.537262343 podStartE2EDuration="3.422102527s" totalImagesPullingTime="884.840184ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:37:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:37:07.420608112 +0000 UTC m=+6053.991957871" watchObservedRunningTime="2026-11-02 13:37:07.422102527 +0000 UTC m=+6053.993452283"
Nov 02 13:37:13.312895 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:13.312863    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:37:14.284884 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:37:14.284852    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh"
Nov 02 13:37:30.984829 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:37:30.984746    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:38:00.984266 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:38:00.984225    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:38:30.984191 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:38:30.984152    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:38:31.080575 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:31.080538    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh"]
Nov 02 13:38:33.956826 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:33.956785    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"]
Nov 02 13:38:33.957370 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:33.957001    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k" podUID="d8b7911a-aa96-4cf8-a47d-67fa2160dfc8" containerName="maas-api" containerID="cri-o://c57ffa9a80df79baa7f1908dfa60bd7da024829440334f5af37e6f28e3be2a97" gracePeriod=30
Nov 02 13:38:35.203637 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.203605    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:38:35.247800 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.247763    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-maas-api-tls\") pod \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\" (UID: \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\") "
Nov 02 13:38:35.247800 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.247802    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-kube-api-access-twfvn\" (UniqueName: \"kubernetes.io/projected/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-kube-api-access-twfvn\") pod \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\" (UID: \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\") "
Nov 02 13:38:35.248059 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.247864    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-openshift-service-ca\") pod \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\" (UID: \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\") "
Nov 02 13:38:35.248330 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.248272    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-openshift-service-ca" pod "d8b7911a-aa96-4cf8-a47d-67fa2160dfc8" (UID: "d8b7911a-aa96-4cf8-a47d-67fa2160dfc8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:38:35.250075 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.250046    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-maas-api-tls" pod "d8b7911a-aa96-4cf8-a47d-67fa2160dfc8" (UID: "d8b7911a-aa96-4cf8-a47d-67fa2160dfc8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:38:35.250177 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.250068    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-kube-api-access-twfvn" pod "d8b7911a-aa96-4cf8-a47d-67fa2160dfc8" (UID: "d8b7911a-aa96-4cf8-a47d-67fa2160dfc8"). InnerVolumeSpecName "kube-api-access-twfvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:38:35.349255 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.349158    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-metrics-certs\") pod \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\" (UID: \"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8\") "
Nov 02 13:38:35.349532 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.349496    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:38:35.349532 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.349518    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:38:35.349740 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.349532    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twfvn\" (UniqueName: \"kubernetes.io/projected/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-kube-api-access-twfvn\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:38:35.351837 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.351801    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-metrics-certs" pod "d8b7911a-aa96-4cf8-a47d-67fa2160dfc8" (UID: "d8b7911a-aa96-4cf8-a47d-67fa2160dfc8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:38:35.450726 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.450687    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:38:35.814195 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.814157    4942 generic.go:356] "Generic (PLEG): container finished" podID="d8b7911a-aa96-4cf8-a47d-67fa2160dfc8" containerID="c57ffa9a80df79baa7f1908dfa60bd7da024829440334f5af37e6f28e3be2a97" exitCode=0
Nov 02 13:38:35.814408 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.814214    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"
Nov 02 13:38:35.814408 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.814248    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k" event={"ID":"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8","Type":"ContainerDied","Data":"c57ffa9a80df79baa7f1908dfa60bd7da024829440334f5af37e6f28e3be2a97"}
Nov 02 13:38:35.814408 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.814347    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k" event={"ID":"d8b7911a-aa96-4cf8-a47d-67fa2160dfc8","Type":"ContainerDied","Data":"2a714363b371c595a21e5c801248251422497cfb0b62a38656b2c346313e3eba"}
Nov 02 13:38:35.814408 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.814373    4942 scope.go:122] "RemoveContainer" containerID="c57ffa9a80df79baa7f1908dfa60bd7da024829440334f5af37e6f28e3be2a97"
Nov 02 13:38:35.826111 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.826085    4942 scope.go:122] "RemoveContainer" containerID="c57ffa9a80df79baa7f1908dfa60bd7da024829440334f5af37e6f28e3be2a97"
Nov 02 13:38:35.826521 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:38:35.826495    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c57ffa9a80df79baa7f1908dfa60bd7da024829440334f5af37e6f28e3be2a97\": container with ID starting with c57ffa9a80df79baa7f1908dfa60bd7da024829440334f5af37e6f28e3be2a97 not found: ID does not exist" containerID="c57ffa9a80df79baa7f1908dfa60bd7da024829440334f5af37e6f28e3be2a97"
Nov 02 13:38:35.826634 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.826533    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c57ffa9a80df79baa7f1908dfa60bd7da024829440334f5af37e6f28e3be2a97"} err="failed to get container status \"c57ffa9a80df79baa7f1908dfa60bd7da024829440334f5af37e6f28e3be2a97\": rpc error: code = NotFound desc = could not find container \"c57ffa9a80df79baa7f1908dfa60bd7da024829440334f5af37e6f28e3be2a97\": container with ID starting with c57ffa9a80df79baa7f1908dfa60bd7da024829440334f5af37e6f28e3be2a97 not found: ID does not exist"
Nov 02 13:38:35.981954 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:35.981910    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"]
Nov 02 13:38:36.031476 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:36.031431    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-7cd5k"]
Nov 02 13:38:36.080991 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:36.080901    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh" podUID="3c454dd8-d004-41c4-a978-c24ff8925ab7" containerName="payload-processing" containerID="cri-o://940fd10b7ab17c3aee50f05754ae073da995ac72e07c49a45409a60f9ed1e126" gracePeriod=25
Nov 02 13:38:37.543315 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:37.543271    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d8b7911a-aa96-4cf8-a47d-67fa2160dfc8" path="/var/lib/kubelet/pods/d8b7911a-aa96-4cf8-a47d-67fa2160dfc8/volumes"
Nov 02 13:38:44.282594 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:44.282555    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh" podUID="3c454dd8-d004-41c4-a978-c24ff8925ab7" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 13:38:54.282000 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:54.281916    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh" podUID="3c454dd8-d004-41c4-a978-c24ff8925ab7" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 02 13:38:56.334587 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:56.334553    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh"
Nov 02 13:38:56.455339 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:56.455275    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c454dd8-d004-41c4-a978-c24ff8925ab7-kube-api-access-5lv7g\" (UniqueName: \"kubernetes.io/projected/3c454dd8-d004-41c4-a978-c24ff8925ab7-kube-api-access-5lv7g\") pod \"3c454dd8-d004-41c4-a978-c24ff8925ab7\" (UID: \"3c454dd8-d004-41c4-a978-c24ff8925ab7\") "
Nov 02 13:38:56.455550 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:56.455373    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c454dd8-d004-41c4-a978-c24ff8925ab7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c454dd8-d004-41c4-a978-c24ff8925ab7-plugins-config-volume\") pod \"3c454dd8-d004-41c4-a978-c24ff8925ab7\" (UID: \"3c454dd8-d004-41c4-a978-c24ff8925ab7\") "
Nov 02 13:38:56.455809 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:56.455780    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c454dd8-d004-41c4-a978-c24ff8925ab7-plugins-config-volume" pod "3c454dd8-d004-41c4-a978-c24ff8925ab7" (UID: "3c454dd8-d004-41c4-a978-c24ff8925ab7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:38:56.457572 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:56.457542    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c454dd8-d004-41c4-a978-c24ff8925ab7-kube-api-access-5lv7g" pod "3c454dd8-d004-41c4-a978-c24ff8925ab7" (UID: "3c454dd8-d004-41c4-a978-c24ff8925ab7"). InnerVolumeSpecName "kube-api-access-5lv7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:38:56.556740 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:56.556703    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lv7g\" (UniqueName: \"kubernetes.io/projected/3c454dd8-d004-41c4-a978-c24ff8925ab7-kube-api-access-5lv7g\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:38:56.556740 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:56.556735    4942 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c454dd8-d004-41c4-a978-c24ff8925ab7-plugins-config-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:38:56.934068 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:56.934031    4942 generic.go:356] "Generic (PLEG): container finished" podID="3c454dd8-d004-41c4-a978-c24ff8925ab7" containerID="940fd10b7ab17c3aee50f05754ae073da995ac72e07c49a45409a60f9ed1e126" exitCode=0
Nov 02 13:38:56.934247 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:56.934076    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh"
Nov 02 13:38:56.934247 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:56.934090    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh" event={"ID":"3c454dd8-d004-41c4-a978-c24ff8925ab7","Type":"ContainerDied","Data":"940fd10b7ab17c3aee50f05754ae073da995ac72e07c49a45409a60f9ed1e126"}
Nov 02 13:38:56.934247 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:56.934137    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh" event={"ID":"3c454dd8-d004-41c4-a978-c24ff8925ab7","Type":"ContainerDied","Data":"514e363961ceca48da7fd687720b6b10ecb11d5f10b0f0885a3b278432c11c29"}
Nov 02 13:38:56.934247 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:56.934160    4942 scope.go:122] "RemoveContainer" containerID="940fd10b7ab17c3aee50f05754ae073da995ac72e07c49a45409a60f9ed1e126"
Nov 02 13:38:56.943698 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:56.943676    4942 scope.go:122] "RemoveContainer" containerID="940fd10b7ab17c3aee50f05754ae073da995ac72e07c49a45409a60f9ed1e126"
Nov 02 13:38:56.943990 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:38:56.943971    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"940fd10b7ab17c3aee50f05754ae073da995ac72e07c49a45409a60f9ed1e126\": container with ID starting with 940fd10b7ab17c3aee50f05754ae073da995ac72e07c49a45409a60f9ed1e126 not found: ID does not exist" containerID="940fd10b7ab17c3aee50f05754ae073da995ac72e07c49a45409a60f9ed1e126"
Nov 02 13:38:56.944035 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:56.943998    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"940fd10b7ab17c3aee50f05754ae073da995ac72e07c49a45409a60f9ed1e126"} err="failed to get container status \"940fd10b7ab17c3aee50f05754ae073da995ac72e07c49a45409a60f9ed1e126\": rpc error: code = NotFound desc = could not find container \"940fd10b7ab17c3aee50f05754ae073da995ac72e07c49a45409a60f9ed1e126\": container with ID starting with 940fd10b7ab17c3aee50f05754ae073da995ac72e07c49a45409a60f9ed1e126 not found: ID does not exist"
Nov 02 13:38:57.098943 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:57.098906    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh"]
Nov 02 13:38:57.104430 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:57.104394    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-7ddf45-567b8bfd6xvrlh"]
Nov 02 13:38:57.542128 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:38:57.542092    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c454dd8-d004-41c4-a978-c24ff8925ab7" path="/var/lib/kubelet/pods/3c454dd8-d004-41c4-a978-c24ff8925ab7/volumes"
Nov 02 13:39:00.982539 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:39:00.982498    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:39:07.961652 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:07.961612    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"]
Nov 02 13:39:07.963258 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:07.963225    4942 cpu_manager.go:414] "containerMap: removing container" podUID="d8b7911a-aa96-4cf8-a47d-67fa2160dfc8" containerName="maas-api"
Nov 02 13:39:07.963462 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:07.963445    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d8b7911a-aa96-4cf8-a47d-67fa2160dfc8" containerName="maas-api"
Nov 02 13:39:07.963628 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:07.963617    4942 cpu_manager.go:414] "containerMap: removing container" podUID="3c454dd8-d004-41c4-a978-c24ff8925ab7" containerName="payload-processing"
Nov 02 13:39:07.963707 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:07.963696    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c454dd8-d004-41c4-a978-c24ff8925ab7" containerName="payload-processing"
Nov 02 13:39:07.963963 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:07.963949    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="d8b7911a-aa96-4cf8-a47d-67fa2160dfc8" containerName="maas-api"
Nov 02 13:39:07.964082 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:07.964071    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c454dd8-d004-41c4-a978-c24ff8925ab7" containerName="payload-processing"
Nov 02 13:39:07.968365 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:07.968342    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:08.022148 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:08.022111    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"]
Nov 02 13:39:08.068538 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:08.068505    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-7ddf45-serving-cert\""
Nov 02 13:39:08.068786 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:08.068562    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pl94d\""
Nov 02 13:39:08.068786 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:08.068571    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 02 13:39:08.075032 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:08.075004    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cf3a44f-6dbc-443f-80dc-7a7732425046-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc\" (UID: \"8cf3a44f-6dbc-443f-80dc-7a7732425046\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:08.075160 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:08.075058    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cf3a44f-6dbc-443f-80dc-7a7732425046-metrics-certs\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc\" (UID: \"8cf3a44f-6dbc-443f-80dc-7a7732425046\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:08.075160 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:08.075077    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cf3a44f-6dbc-443f-80dc-7a7732425046-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc\" (UID: \"8cf3a44f-6dbc-443f-80dc-7a7732425046\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:08.075275 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:08.075250    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mzmn\" (UniqueName: \"kubernetes.io/projected/8cf3a44f-6dbc-443f-80dc-7a7732425046-kube-api-access-6mzmn\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc\" (UID: \"8cf3a44f-6dbc-443f-80dc-7a7732425046\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:08.176275 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:08.176230    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mzmn\" (UniqueName: \"kubernetes.io/projected/8cf3a44f-6dbc-443f-80dc-7a7732425046-kube-api-access-6mzmn\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc\" (UID: \"8cf3a44f-6dbc-443f-80dc-7a7732425046\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:08.176499 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:08.176347    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cf3a44f-6dbc-443f-80dc-7a7732425046-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc\" (UID: \"8cf3a44f-6dbc-443f-80dc-7a7732425046\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:08.176499 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:08.176400    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cf3a44f-6dbc-443f-80dc-7a7732425046-metrics-certs\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc\" (UID: \"8cf3a44f-6dbc-443f-80dc-7a7732425046\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:08.176499 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:08.176426    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cf3a44f-6dbc-443f-80dc-7a7732425046-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc\" (UID: \"8cf3a44f-6dbc-443f-80dc-7a7732425046\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:08.176688 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:39:08.176505    4942 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-serving-cert: secret "maas-api-e2e-worker-single-main-7ddf45-serving-cert" not found
Nov 02 13:39:08.176688 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:39:08.176585    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cf3a44f-6dbc-443f-80dc-7a7732425046-maas-api-tls podName:8cf3a44f-6dbc-443f-80dc-7a7732425046 nodeName:}" failed. No retries permitted until 2026-11-02 13:39:08.676563723 +0000 UTC m=+6175.247913465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8cf3a44f-6dbc-443f-80dc-7a7732425046-maas-api-tls") pod "maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc" (UID: "8cf3a44f-6dbc-443f-80dc-7a7732425046") : secret "maas-api-e2e-worker-single-main-7ddf45-serving-cert" not found
Nov 02 13:39:08.177093 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:08.177069    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cf3a44f-6dbc-443f-80dc-7a7732425046-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc\" (UID: \"8cf3a44f-6dbc-443f-80dc-7a7732425046\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:08.178904 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:08.178878    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cf3a44f-6dbc-443f-80dc-7a7732425046-metrics-certs\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc\" (UID: \"8cf3a44f-6dbc-443f-80dc-7a7732425046\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:08.240526 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:08.240445    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mzmn\" (UniqueName: \"kubernetes.io/projected/8cf3a44f-6dbc-443f-80dc-7a7732425046-kube-api-access-6mzmn\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc\" (UID: \"8cf3a44f-6dbc-443f-80dc-7a7732425046\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:08.682418 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:08.682380    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cf3a44f-6dbc-443f-80dc-7a7732425046-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc\" (UID: \"8cf3a44f-6dbc-443f-80dc-7a7732425046\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:08.684819 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:08.684790    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cf3a44f-6dbc-443f-80dc-7a7732425046-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc\" (UID: \"8cf3a44f-6dbc-443f-80dc-7a7732425046\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:08.881643 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:08.881580    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:09.260481 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:09.260449    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"]
Nov 02 13:39:09.264739 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:39:09.264538    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cf3a44f_6dbc_443f_80dc_7a7732425046.slice/crio-d887d7b342b54e04a7e458f30a7a31e2c140e160b13ed05be1b4266f8fa8a8cd WatchSource:0}: Error finding container d887d7b342b54e04a7e458f30a7a31e2c140e160b13ed05be1b4266f8fa8a8cd: Status 404 returned error can't find the container with id d887d7b342b54e04a7e458f30a7a31e2c140e160b13ed05be1b4266f8fa8a8cd
Nov 02 13:39:10.005672 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:10.005634    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc" event={"ID":"8cf3a44f-6dbc-443f-80dc-7a7732425046","Type":"ContainerStarted","Data":"d887d7b342b54e04a7e458f30a7a31e2c140e160b13ed05be1b4266f8fa8a8cd"}
Nov 02 13:39:11.013172 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:11.013132    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc" event={"ID":"8cf3a44f-6dbc-443f-80dc-7a7732425046","Type":"ContainerStarted","Data":"c4caa13a80ee7eb4ee2231b232f6480eb32c8c6ebd94baf76b57838e25fefffe"}
Nov 02 13:39:11.013613 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:11.013217    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:11.134508 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:11.134447    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc" podStartSLOduration=3.232778823 podStartE2EDuration="4.134431323s" totalImagesPullingTime="901.6525ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:39:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:39:11.132032992 +0000 UTC m=+6177.703382761" watchObservedRunningTime="2026-11-02 13:39:11.134431323 +0000 UTC m=+6177.705781099"
Nov 02 13:39:17.026469 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:17.026439    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:30.982470 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:39:30.982430    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:39:37.389422 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:37.389380    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"]
Nov 02 13:39:37.389842 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:37.389603    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc" podUID="8cf3a44f-6dbc-443f-80dc-7a7732425046" containerName="maas-api" containerID="cri-o://c4caa13a80ee7eb4ee2231b232f6480eb32c8c6ebd94baf76b57838e25fefffe" gracePeriod=30
Nov 02 13:39:37.646081 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:37.646018    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:37.784866 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:37.784809    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8cf3a44f-6dbc-443f-80dc-7a7732425046-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cf3a44f-6dbc-443f-80dc-7a7732425046-maas-api-tls\") pod \"8cf3a44f-6dbc-443f-80dc-7a7732425046\" (UID: \"8cf3a44f-6dbc-443f-80dc-7a7732425046\") "
Nov 02 13:39:37.785065 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:37.784992    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8cf3a44f-6dbc-443f-80dc-7a7732425046-kube-api-access-6mzmn\" (UniqueName: \"kubernetes.io/projected/8cf3a44f-6dbc-443f-80dc-7a7732425046-kube-api-access-6mzmn\") pod \"8cf3a44f-6dbc-443f-80dc-7a7732425046\" (UID: \"8cf3a44f-6dbc-443f-80dc-7a7732425046\") "
Nov 02 13:39:37.785065 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:37.785031    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8cf3a44f-6dbc-443f-80dc-7a7732425046-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cf3a44f-6dbc-443f-80dc-7a7732425046-openshift-service-ca\") pod \"8cf3a44f-6dbc-443f-80dc-7a7732425046\" (UID: \"8cf3a44f-6dbc-443f-80dc-7a7732425046\") "
Nov 02 13:39:37.785065 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:37.785062    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8cf3a44f-6dbc-443f-80dc-7a7732425046-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cf3a44f-6dbc-443f-80dc-7a7732425046-metrics-certs\") pod \"8cf3a44f-6dbc-443f-80dc-7a7732425046\" (UID: \"8cf3a44f-6dbc-443f-80dc-7a7732425046\") "
Nov 02 13:39:37.785604 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:37.785570    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cf3a44f-6dbc-443f-80dc-7a7732425046-openshift-service-ca" pod "8cf3a44f-6dbc-443f-80dc-7a7732425046" (UID: "8cf3a44f-6dbc-443f-80dc-7a7732425046"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:39:37.787191 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:37.787151    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cf3a44f-6dbc-443f-80dc-7a7732425046-maas-api-tls" pod "8cf3a44f-6dbc-443f-80dc-7a7732425046" (UID: "8cf3a44f-6dbc-443f-80dc-7a7732425046"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:39:37.787325 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:37.787202    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cf3a44f-6dbc-443f-80dc-7a7732425046-kube-api-access-6mzmn" pod "8cf3a44f-6dbc-443f-80dc-7a7732425046" (UID: "8cf3a44f-6dbc-443f-80dc-7a7732425046"). InnerVolumeSpecName "kube-api-access-6mzmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:39:37.787325 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:37.787275    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cf3a44f-6dbc-443f-80dc-7a7732425046-metrics-certs" pod "8cf3a44f-6dbc-443f-80dc-7a7732425046" (UID: "8cf3a44f-6dbc-443f-80dc-7a7732425046"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:39:37.886519 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:37.886481    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cf3a44f-6dbc-443f-80dc-7a7732425046-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:39:37.886519 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:37.886514    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mzmn\" (UniqueName: \"kubernetes.io/projected/8cf3a44f-6dbc-443f-80dc-7a7732425046-kube-api-access-6mzmn\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:39:37.886519 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:37.886525    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cf3a44f-6dbc-443f-80dc-7a7732425046-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:39:37.886766 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:37.886534    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8cf3a44f-6dbc-443f-80dc-7a7732425046-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:39:38.160857 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:38.160812    4942 generic.go:356] "Generic (PLEG): container finished" podID="8cf3a44f-6dbc-443f-80dc-7a7732425046" containerID="c4caa13a80ee7eb4ee2231b232f6480eb32c8c6ebd94baf76b57838e25fefffe" exitCode=0
Nov 02 13:39:38.161064 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:38.160887    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"
Nov 02 13:39:38.161064 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:38.160897    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc" event={"ID":"8cf3a44f-6dbc-443f-80dc-7a7732425046","Type":"ContainerDied","Data":"c4caa13a80ee7eb4ee2231b232f6480eb32c8c6ebd94baf76b57838e25fefffe"}
Nov 02 13:39:38.161064 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:38.160986    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc" event={"ID":"8cf3a44f-6dbc-443f-80dc-7a7732425046","Type":"ContainerDied","Data":"d887d7b342b54e04a7e458f30a7a31e2c140e160b13ed05be1b4266f8fa8a8cd"}
Nov 02 13:39:38.161064 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:38.161016    4942 scope.go:122] "RemoveContainer" containerID="c4caa13a80ee7eb4ee2231b232f6480eb32c8c6ebd94baf76b57838e25fefffe"
Nov 02 13:39:38.170809 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:38.170777    4942 scope.go:122] "RemoveContainer" containerID="c4caa13a80ee7eb4ee2231b232f6480eb32c8c6ebd94baf76b57838e25fefffe"
Nov 02 13:39:38.171104 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:39:38.171085    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4caa13a80ee7eb4ee2231b232f6480eb32c8c6ebd94baf76b57838e25fefffe\": container with ID starting with c4caa13a80ee7eb4ee2231b232f6480eb32c8c6ebd94baf76b57838e25fefffe not found: ID does not exist" containerID="c4caa13a80ee7eb4ee2231b232f6480eb32c8c6ebd94baf76b57838e25fefffe"
Nov 02 13:39:38.171197 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:38.171113    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4caa13a80ee7eb4ee2231b232f6480eb32c8c6ebd94baf76b57838e25fefffe"} err="failed to get container status \"c4caa13a80ee7eb4ee2231b232f6480eb32c8c6ebd94baf76b57838e25fefffe\": rpc error: code = NotFound desc = could not find container \"c4caa13a80ee7eb4ee2231b232f6480eb32c8c6ebd94baf76b57838e25fefffe\": container with ID starting with c4caa13a80ee7eb4ee2231b232f6480eb32c8c6ebd94baf76b57838e25fefffe not found: ID does not exist"
Nov 02 13:39:38.326728 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:38.326690    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"]
Nov 02 13:39:38.376981 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:38.376947    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-7ddf45-8fc45c7bd-jdzqc"]
Nov 02 13:39:39.543025 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:39.542991    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8cf3a44f-6dbc-443f-80dc-7a7732425046" path="/var/lib/kubelet/pods/8cf3a44f-6dbc-443f-80dc-7a7732425046/volumes"
Nov 02 13:39:58.422486 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:58.422429    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-pjqbs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:39:58.423178 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:58.423134    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs" podUID="d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:39:58.445486 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:58.445437    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-qzbzn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 13:39:58.445833 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:39:58.445802    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn" podUID="d43b8d98-489b-4c47-8718-9d47509709b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 13:40:00.983584 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:40:00.983547    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:40:02.525468 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:40:02.525429    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc"]
Nov 02 13:40:02.525941 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:40:02.525639    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc" podUID="fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902" containerName="manager" containerID="cri-o://3a96408d7e9891ddd779fe7e0b9a6dd9b637b10e7c2c1e57703494720d3e08bf" gracePeriod=10
Nov 02 13:40:05.350840 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:40:05.350803    4942 generic.go:356] "Generic (PLEG): container finished" podID="fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902" containerID="3a96408d7e9891ddd779fe7e0b9a6dd9b637b10e7c2c1e57703494720d3e08bf" exitCode=0
Nov 02 13:40:05.351333 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:40:05.350931    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc" event={"ID":"fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902","Type":"ContainerDied","Data":"3a96408d7e9891ddd779fe7e0b9a6dd9b637b10e7c2c1e57703494720d3e08bf"}
Nov 02 13:40:05.473831 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:40:05.473806    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc"
Nov 02 13:40:05.603725 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:40:05.603690    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902-extensions-socket-volume\") pod \"fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902\" (UID: \"fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902\") "
Nov 02 13:40:05.603903 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:40:05.603799    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902-kube-api-access-6555j\" (UniqueName: \"kubernetes.io/projected/fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902-kube-api-access-6555j\") pod \"fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902\" (UID: \"fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902\") "
Nov 02 13:40:05.604098 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:40:05.604068    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902-extensions-socket-volume" pod "fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902" (UID: "fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:40:05.604371 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:40:05.604348    4942 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902-extensions-socket-volume\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:40:05.606010 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:40:05.605982    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902-kube-api-access-6555j" pod "fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902" (UID: "fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902"). InnerVolumeSpecName "kube-api-access-6555j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:40:05.705382 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:40:05.705345    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6555j\" (UniqueName: \"kubernetes.io/projected/fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902-kube-api-access-6555j\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:40:06.357937 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:40:06.357905    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc"
Nov 02 13:40:06.358436 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:40:06.357905    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc" event={"ID":"fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902","Type":"ContainerDied","Data":"ae7f3f0945b0b863a0c61e83214204d5ce8c43330d2f08e683b856226ed600d0"}
Nov 02 13:40:06.358436 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:40:06.358056    4942 scope.go:122] "RemoveContainer" containerID="3a96408d7e9891ddd779fe7e0b9a6dd9b637b10e7c2c1e57703494720d3e08bf"
Nov 02 13:40:06.572363 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:40:06.572319    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc"]
Nov 02 13:40:06.622249 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:40:06.622167    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-7wrsc"]
Nov 02 13:40:07.542102 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:40:07.542069    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902" path="/var/lib/kubelet/pods/fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902/volumes"
Nov 02 13:40:30.984333 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:40:30.984223    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:41:00.983338 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:41:00.983278    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:41:06.146931 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:06.146895    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-bsm7c"]
Nov 02 13:41:06.147536 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:06.147513    4942 cpu_manager.go:414] "containerMap: removing container" podUID="8cf3a44f-6dbc-443f-80dc-7a7732425046" containerName="maas-api"
Nov 02 13:41:06.147633 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:06.147538    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8cf3a44f-6dbc-443f-80dc-7a7732425046" containerName="maas-api"
Nov 02 13:41:06.147633 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:06.147579    4942 cpu_manager.go:414] "containerMap: removing container" podUID="fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902" containerName="manager"
Nov 02 13:41:06.147633 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:06.147588    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902" containerName="manager"
Nov 02 13:41:06.147795 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:06.147708    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="fcf7d774-bb8f-4ee4-8a74-4fb7e7ebd902" containerName="manager"
Nov 02 13:41:06.147795 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:06.147729    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="8cf3a44f-6dbc-443f-80dc-7a7732425046" containerName="maas-api"
Nov 02 13:41:06.150773 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:06.150749    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-bsm7c"
Nov 02 13:41:06.202669 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:06.202630    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-bsm7c"]
Nov 02 13:41:06.249268 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:06.249225    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gzmrw\""
Nov 02 13:41:06.297275 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:06.297242    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f77818a-ad1b-48ec-b8aa-76fce65162f6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-bsm7c\" (UID: \"4f77818a-ad1b-48ec-b8aa-76fce65162f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-bsm7c"
Nov 02 13:41:06.297460 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:06.297277    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxw5l\" (UniqueName: \"kubernetes.io/projected/4f77818a-ad1b-48ec-b8aa-76fce65162f6-kube-api-access-lxw5l\") pod \"kuadrant-operator-controller-manager-678d87766f-bsm7c\" (UID: \"4f77818a-ad1b-48ec-b8aa-76fce65162f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-bsm7c"
Nov 02 13:41:06.398502 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:06.398392    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f77818a-ad1b-48ec-b8aa-76fce65162f6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-bsm7c\" (UID: \"4f77818a-ad1b-48ec-b8aa-76fce65162f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-bsm7c"
Nov 02 13:41:06.398502 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:06.398442    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxw5l\" (UniqueName: \"kubernetes.io/projected/4f77818a-ad1b-48ec-b8aa-76fce65162f6-kube-api-access-lxw5l\") pod \"kuadrant-operator-controller-manager-678d87766f-bsm7c\" (UID: \"4f77818a-ad1b-48ec-b8aa-76fce65162f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-bsm7c"
Nov 02 13:41:06.398861 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:06.398831    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f77818a-ad1b-48ec-b8aa-76fce65162f6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-bsm7c\" (UID: \"4f77818a-ad1b-48ec-b8aa-76fce65162f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-bsm7c"
Nov 02 13:41:06.460252 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:06.460216    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxw5l\" (UniqueName: \"kubernetes.io/projected/4f77818a-ad1b-48ec-b8aa-76fce65162f6-kube-api-access-lxw5l\") pod \"kuadrant-operator-controller-manager-678d87766f-bsm7c\" (UID: \"4f77818a-ad1b-48ec-b8aa-76fce65162f6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-bsm7c"
Nov 02 13:41:06.465076 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:06.465042    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-bsm7c"
Nov 02 13:41:06.946025 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:06.945997    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-bsm7c"]
Nov 02 13:41:06.948858 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:41:06.948820    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f77818a_ad1b_48ec_b8aa_76fce65162f6.slice/crio-d65072bdc57e5727c72ea84f909e4915b7efdc6f7e6712140bd2af10794b8aa7 WatchSource:0}: Error finding container d65072bdc57e5727c72ea84f909e4915b7efdc6f7e6712140bd2af10794b8aa7: Status 404 returned error can't find the container with id d65072bdc57e5727c72ea84f909e4915b7efdc6f7e6712140bd2af10794b8aa7
Nov 02 13:41:07.692774 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:07.692739    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-bsm7c" event={"ID":"4f77818a-ad1b-48ec-b8aa-76fce65162f6","Type":"ContainerStarted","Data":"276654dcddfe862945fb4c5b0d3527e9442b9621fbcfc7b3b5291ed571810b73"}
Nov 02 13:41:07.693237 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:07.692792    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-bsm7c" event={"ID":"4f77818a-ad1b-48ec-b8aa-76fce65162f6","Type":"ContainerStarted","Data":"d65072bdc57e5727c72ea84f909e4915b7efdc6f7e6712140bd2af10794b8aa7"}
Nov 02 13:41:07.693237 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:07.692875    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-bsm7c"
Nov 02 13:41:07.915364 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:07.915281    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-bsm7c" podStartSLOduration=1.915267768 podStartE2EDuration="1.915267768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:41:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:41:07.863607982 +0000 UTC m=+6294.434957739" watchObservedRunningTime="2026-11-02 13:41:07.915267768 +0000 UTC m=+6294.486617524"
Nov 02 13:41:14.395435 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:14.395402    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:41:14.418986 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:14.418962    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:41:18.700649 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:41:18.700616    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-bsm7c"
Nov 02 13:41:31.152391 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:41:31.152147    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:42:00.983239 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:42:00.983160    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:42:07.270620 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.270581    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"]
Nov 02 13:42:07.274429 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.274404    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"
Nov 02 13:42:07.288600 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.288578    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6459565456-l82jx"]
Nov 02 13:42:07.291756 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.291740    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx"
Nov 02 13:42:07.335646 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.335617    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pl94d\""
Nov 02 13:42:07.335764 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.335653    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 02 13:42:07.376018 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.375989    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 02 13:42:07.378843 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.378817    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"]
Nov 02 13:42:07.383332 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.383287    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6459565456-l82jx"]
Nov 02 13:42:07.442091 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.442065    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djw5j\" (UniqueName: \"kubernetes.io/projected/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-kube-api-access-djw5j\") pod \"maas-api-6866d668fb-6kfzr\" (UID: \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\") " pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"
Nov 02 13:42:07.442215 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.442099    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32d13fe9-ef8b-4050-8b97-f52f6035c1d9-metrics-certs\") pod \"maas-api-6459565456-l82jx\" (UID: \"32d13fe9-ef8b-4050-8b97-f52f6035c1d9\") " pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx"
Nov 02 13:42:07.442215 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.442126    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-openshift-service-ca\") pod \"maas-api-6866d668fb-6kfzr\" (UID: \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\") " pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"
Nov 02 13:42:07.442215 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.442177    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32d13fe9-ef8b-4050-8b97-f52f6035c1d9-maas-api-tls\") pod \"maas-api-6459565456-l82jx\" (UID: \"32d13fe9-ef8b-4050-8b97-f52f6035c1d9\") " pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx"
Nov 02 13:42:07.442355 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.442265    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-maas-api-tls\") pod \"maas-api-6866d668fb-6kfzr\" (UID: \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\") " pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"
Nov 02 13:42:07.442355 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.442290    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32d13fe9-ef8b-4050-8b97-f52f6035c1d9-openshift-service-ca\") pod \"maas-api-6459565456-l82jx\" (UID: \"32d13fe9-ef8b-4050-8b97-f52f6035c1d9\") " pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx"
Nov 02 13:42:07.442355 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.442339    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-metrics-certs\") pod \"maas-api-6866d668fb-6kfzr\" (UID: \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\") " pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"
Nov 02 13:42:07.442466 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.442382    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j822\" (UniqueName: \"kubernetes.io/projected/32d13fe9-ef8b-4050-8b97-f52f6035c1d9-kube-api-access-5j822\") pod \"maas-api-6459565456-l82jx\" (UID: \"32d13fe9-ef8b-4050-8b97-f52f6035c1d9\") " pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx"
Nov 02 13:42:07.543703 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.543624    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-maas-api-tls\") pod \"maas-api-6866d668fb-6kfzr\" (UID: \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\") " pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"
Nov 02 13:42:07.543703 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.543665    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32d13fe9-ef8b-4050-8b97-f52f6035c1d9-openshift-service-ca\") pod \"maas-api-6459565456-l82jx\" (UID: \"32d13fe9-ef8b-4050-8b97-f52f6035c1d9\") " pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx"
Nov 02 13:42:07.543703 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.543687    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-metrics-certs\") pod \"maas-api-6866d668fb-6kfzr\" (UID: \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\") " pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"
Nov 02 13:42:07.543980 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.543739    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j822\" (UniqueName: \"kubernetes.io/projected/32d13fe9-ef8b-4050-8b97-f52f6035c1d9-kube-api-access-5j822\") pod \"maas-api-6459565456-l82jx\" (UID: \"32d13fe9-ef8b-4050-8b97-f52f6035c1d9\") " pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx"
Nov 02 13:42:07.543980 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.543822    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djw5j\" (UniqueName: \"kubernetes.io/projected/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-kube-api-access-djw5j\") pod \"maas-api-6866d668fb-6kfzr\" (UID: \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\") " pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"
Nov 02 13:42:07.543980 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.543852    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32d13fe9-ef8b-4050-8b97-f52f6035c1d9-metrics-certs\") pod \"maas-api-6459565456-l82jx\" (UID: \"32d13fe9-ef8b-4050-8b97-f52f6035c1d9\") " pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx"
Nov 02 13:42:07.543980 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.543889    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-openshift-service-ca\") pod \"maas-api-6866d668fb-6kfzr\" (UID: \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\") " pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"
Nov 02 13:42:07.543980 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.543912    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32d13fe9-ef8b-4050-8b97-f52f6035c1d9-maas-api-tls\") pod \"maas-api-6459565456-l82jx\" (UID: \"32d13fe9-ef8b-4050-8b97-f52f6035c1d9\") " pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx"
Nov 02 13:42:07.545178 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.545139    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/32d13fe9-ef8b-4050-8b97-f52f6035c1d9-openshift-service-ca\") pod \"maas-api-6459565456-l82jx\" (UID: \"32d13fe9-ef8b-4050-8b97-f52f6035c1d9\") " pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx"
Nov 02 13:42:07.545543 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.545509    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-openshift-service-ca\") pod \"maas-api-6866d668fb-6kfzr\" (UID: \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\") " pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"
Nov 02 13:42:07.557401 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.548073    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/32d13fe9-ef8b-4050-8b97-f52f6035c1d9-maas-api-tls\") pod \"maas-api-6459565456-l82jx\" (UID: \"32d13fe9-ef8b-4050-8b97-f52f6035c1d9\") " pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx"
Nov 02 13:42:07.557401 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.548118    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-metrics-certs\") pod \"maas-api-6866d668fb-6kfzr\" (UID: \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\") " pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"
Nov 02 13:42:07.559042 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.559014    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/32d13fe9-ef8b-4050-8b97-f52f6035c1d9-metrics-certs\") pod \"maas-api-6459565456-l82jx\" (UID: \"32d13fe9-ef8b-4050-8b97-f52f6035c1d9\") " pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx"
Nov 02 13:42:07.559583 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.559559    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-maas-api-tls\") pod \"maas-api-6866d668fb-6kfzr\" (UID: \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\") " pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"
Nov 02 13:42:07.617780 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.617755    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j822\" (UniqueName: \"kubernetes.io/projected/32d13fe9-ef8b-4050-8b97-f52f6035c1d9-kube-api-access-5j822\") pod \"maas-api-6459565456-l82jx\" (UID: \"32d13fe9-ef8b-4050-8b97-f52f6035c1d9\") " pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx"
Nov 02 13:42:07.617780 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.617772    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djw5j\" (UniqueName: \"kubernetes.io/projected/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-kube-api-access-djw5j\") pod \"maas-api-6866d668fb-6kfzr\" (UID: \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\") " pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"
Nov 02 13:42:07.888214 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.888181    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"
Nov 02 13:42:07.904288 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:07.904264    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx"
Nov 02 13:42:08.318146 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:08.318122    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6459565456-l82jx"]
Nov 02 13:42:08.319262 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:42:08.319234    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32d13fe9_ef8b_4050_8b97_f52f6035c1d9.slice/crio-bffd62a83f727cf058f88792c9308935d0b2547608020a9dc91b96dddf8c8daa WatchSource:0}: Error finding container bffd62a83f727cf058f88792c9308935d0b2547608020a9dc91b96dddf8c8daa: Status 404 returned error can't find the container with id bffd62a83f727cf058f88792c9308935d0b2547608020a9dc91b96dddf8c8daa
Nov 02 13:42:08.320572 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:08.320555    4942 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 13:42:08.357207 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:42:08.357183    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55bb3e61_cc4a_48ec_a5ad_c3d97233b9e3.slice/crio-868d135ffac77ae21b66c7b07fc2d52461070dcef3e2f53da964ffd558696328 WatchSource:0}: Error finding container 868d135ffac77ae21b66c7b07fc2d52461070dcef3e2f53da964ffd558696328: Status 404 returned error can't find the container with id 868d135ffac77ae21b66c7b07fc2d52461070dcef3e2f53da964ffd558696328
Nov 02 13:42:08.368955 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:08.368924    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"]
Nov 02 13:42:09.118443 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:09.118401    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx" event={"ID":"32d13fe9-ef8b-4050-8b97-f52f6035c1d9","Type":"ContainerStarted","Data":"bffd62a83f727cf058f88792c9308935d0b2547608020a9dc91b96dddf8c8daa"}
Nov 02 13:42:09.119825 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:09.119794    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr" event={"ID":"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3","Type":"ContainerStarted","Data":"868d135ffac77ae21b66c7b07fc2d52461070dcef3e2f53da964ffd558696328"}
Nov 02 13:42:10.126698 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:10.126656    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx" event={"ID":"32d13fe9-ef8b-4050-8b97-f52f6035c1d9","Type":"ContainerStarted","Data":"d9795438adaa1d9d7b08e318d40a9931dfe126af9c9a427fee3a7a4a16270bbe"}
Nov 02 13:42:10.127145 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:10.126845    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx"
Nov 02 13:42:10.128348 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:10.128321    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr" event={"ID":"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3","Type":"ContainerStarted","Data":"99dd6b534af9ac4029bc54dcefa8155cad08175969b0f8ce0c0f23900165d58d"}
Nov 02 13:42:10.128506 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:10.128420    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"
Nov 02 13:42:10.246282 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:10.246236    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx" podStartSLOduration=2.357069316 podStartE2EDuration="3.246223082s" totalImagesPullingTime="889.153766ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:42:10.243884974 +0000 UTC m=+6356.815234731" watchObservedRunningTime="2026-11-02 13:42:10.246223082 +0000 UTC m=+6356.817572878"
Nov 02 13:42:10.362311 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:10.362252    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr" podStartSLOduration=2.43418235 podStartE2EDuration="3.362240513s" totalImagesPullingTime="928.058163ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:42:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:42:10.3612776 +0000 UTC m=+6356.932627361" watchObservedRunningTime="2026-11-02 13:42:10.362240513 +0000 UTC m=+6356.933590270"
Nov 02 13:42:16.139943 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:16.139909    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"
Nov 02 13:42:16.140462 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:16.140436    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6459565456-l82jx"
Nov 02 13:42:16.411911 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:16.411825    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"]
Nov 02 13:42:16.412079 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:16.412046    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr" podUID="55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3" containerName="maas-api" containerID="cri-o://99dd6b534af9ac4029bc54dcefa8155cad08175969b0f8ce0c0f23900165d58d" gracePeriod=30
Nov 02 13:42:16.659617 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:16.659592    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"
Nov 02 13:42:16.830612 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:16.830530    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-metrics-certs\") pod \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\" (UID: \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\") "
Nov 02 13:42:16.830755 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:16.830666    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-openshift-service-ca\") pod \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\" (UID: \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\") "
Nov 02 13:42:16.830755 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:16.830707    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-kube-api-access-djw5j\" (UniqueName: \"kubernetes.io/projected/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-kube-api-access-djw5j\") pod \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\" (UID: \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\") "
Nov 02 13:42:16.830755 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:16.830731    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-maas-api-tls\") pod \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\" (UID: \"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3\") "
Nov 02 13:42:16.831025 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:16.831000    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-openshift-service-ca" pod "55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3" (UID: "55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:42:16.832834 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:16.832804    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-maas-api-tls" pod "55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3" (UID: "55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:42:16.832834 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:16.832820    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-metrics-certs" pod "55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3" (UID: "55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:42:16.832834 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:16.832825    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-kube-api-access-djw5j" pod "55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3" (UID: "55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3"). InnerVolumeSpecName "kube-api-access-djw5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:42:16.932529 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:16.932498    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:42:16.932529 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:16.932524    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:42:16.932529 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:16.932535    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djw5j\" (UniqueName: \"kubernetes.io/projected/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-kube-api-access-djw5j\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:42:16.932761 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:16.932546    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:42:17.170001 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:17.169967    4942 generic.go:356] "Generic (PLEG): container finished" podID="55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3" containerID="99dd6b534af9ac4029bc54dcefa8155cad08175969b0f8ce0c0f23900165d58d" exitCode=0
Nov 02 13:42:17.170457 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:17.170016    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr" event={"ID":"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3","Type":"ContainerDied","Data":"99dd6b534af9ac4029bc54dcefa8155cad08175969b0f8ce0c0f23900165d58d"}
Nov 02 13:42:17.170457 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:17.170031    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"
Nov 02 13:42:17.170457 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:17.170056    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr" event={"ID":"55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3","Type":"ContainerDied","Data":"868d135ffac77ae21b66c7b07fc2d52461070dcef3e2f53da964ffd558696328"}
Nov 02 13:42:17.170457 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:17.170089    4942 scope.go:122] "RemoveContainer" containerID="99dd6b534af9ac4029bc54dcefa8155cad08175969b0f8ce0c0f23900165d58d"
Nov 02 13:42:17.181059 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:17.180849    4942 scope.go:122] "RemoveContainer" containerID="99dd6b534af9ac4029bc54dcefa8155cad08175969b0f8ce0c0f23900165d58d"
Nov 02 13:42:17.181252 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:42:17.181223    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99dd6b534af9ac4029bc54dcefa8155cad08175969b0f8ce0c0f23900165d58d\": container with ID starting with 99dd6b534af9ac4029bc54dcefa8155cad08175969b0f8ce0c0f23900165d58d not found: ID does not exist" containerID="99dd6b534af9ac4029bc54dcefa8155cad08175969b0f8ce0c0f23900165d58d"
Nov 02 13:42:17.181386 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:17.181262    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99dd6b534af9ac4029bc54dcefa8155cad08175969b0f8ce0c0f23900165d58d"} err="failed to get container status \"99dd6b534af9ac4029bc54dcefa8155cad08175969b0f8ce0c0f23900165d58d\": rpc error: code = NotFound desc = could not find container \"99dd6b534af9ac4029bc54dcefa8155cad08175969b0f8ce0c0f23900165d58d\": container with ID starting with 99dd6b534af9ac4029bc54dcefa8155cad08175969b0f8ce0c0f23900165d58d not found: ID does not exist"
Nov 02 13:42:17.335752 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:17.335703    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"]
Nov 02 13:42:17.392500 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:17.386688    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6866d668fb-6kfzr"]
Nov 02 13:42:17.543268 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:42:17.543192    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3" path="/var/lib/kubelet/pods/55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3/volumes"
Nov 02 13:42:30.983038 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:42:30.983003    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:43:00.982576 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:43:00.982537    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:43:30.983005 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:43:30.982920    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:44:00.982669 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:44:00.982629    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:44:30.982955 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:44:30.982902    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:45:00.982385 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:45:00.982273    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:45:30.982966 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:45:30.982929    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:46:00.982477 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:46:00.982442    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:46:14.433105 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:46:14.433078    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:46:14.455478 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:46:14.455453    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:46:30.982674 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:46:30.982585    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:47:00.983590 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:47:00.983555    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:47:30.981624 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:47:30.981589    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:47:51.942633 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:51.942598    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"]
Nov 02 13:47:51.943218 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:51.942799    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs" podUID="d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" containerName="istio-proxy" containerID="cri-o://cbd95db86fefff5631431a68c4a1902eaac0b4a6cf4f5bca717f3cfabefe157a" gracePeriod=30
Nov 02 13:47:51.992146 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:51.992109    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"]
Nov 02 13:47:51.992343 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:51.992321    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" containerName="istio-proxy" containerID="cri-o://7c7ca367d19d30279ac69503d1406d6f31429a22e4577d425f67f71ac45f4cba" gracePeriod=30
Nov 02 13:47:51.994264 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:51.994233    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"]
Nov 02 13:47:51.994453 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:51.994432    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn" podUID="d43b8d98-489b-4c47-8718-9d47509709b5" containerName="istio-proxy" containerID="cri-o://ad98961077edd5d62663d02aedda2cf80ea5c9da68de1d2709c4241b18a8a5da" gracePeriod=30
Nov 02 13:47:52.041970 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:52.041937    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"]
Nov 02 13:47:52.042172 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:52.042137    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy" containerID="cri-o://815996b2c876bde0ef3067780939afb791d433f7c977d5dbbf3ac039faf10826" gracePeriod=30
Nov 02 13:47:54.830221 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:54.830134    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-wkxz9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 02 13:47:54.830620 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:54.830224    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 02 13:47:57.245902 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.245870    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:47:57.321278 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.321249    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:47:57.334342 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.334312    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-credential-socket\") pod \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") "
Nov 02 13:47:57.334518 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.334352    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-data\") pod \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") "
Nov 02 13:47:57.334518 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.334397    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-workload-certs\") pod \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") "
Nov 02 13:47:57.334518 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.334421    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-workload-socket\") pod \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") "
Nov 02 13:47:57.334518 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.334488    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istiod-ca-cert\") pod \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") "
Nov 02 13:47:57.334518 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.334518    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-token\" (UniqueName: \"kubernetes.io/projected/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-token\") pod \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") "
Nov 02 13:47:57.334794 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.334562    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-envoy\") pod \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") "
Nov 02 13:47:57.334794 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.334612    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-kube-api-access-xl6cp\" (UniqueName: \"kubernetes.io/projected/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-kube-api-access-xl6cp\") pod \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") "
Nov 02 13:47:57.334794 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.334614    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-credential-socket" pod "d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" (UID: "d1c1a136-bb43-4e4d-b47b-3d7c2af690ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.334794 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.334645    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-podinfo\") pod \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\" (UID: \"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca\") "
Nov 02 13:47:57.335002 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.334837    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-data" pod "d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" (UID: "d1c1a136-bb43-4e4d-b47b-3d7c2af690ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.335107 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.335048    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istiod-ca-cert" pod "d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" (UID: "d1c1a136-bb43-4e4d-b47b-3d7c2af690ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:47:57.335209 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.335128    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.335209 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.335145    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.335513 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.335482    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-workload-socket" pod "d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" (UID: "d1c1a136-bb43-4e4d-b47b-3d7c2af690ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.335622 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.335549    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-workload-certs" pod "d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" (UID: "d1c1a136-bb43-4e4d-b47b-3d7c2af690ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.338006 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.337983    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-envoy" pod "d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" (UID: "d1c1a136-bb43-4e4d-b47b-3d7c2af690ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.340044 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.340019    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-podinfo" pod "d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" (UID: "d1c1a136-bb43-4e4d-b47b-3d7c2af690ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 13:47:57.340137 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.340118    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-token" pod "d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" (UID: "d1c1a136-bb43-4e4d-b47b-3d7c2af690ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:47:57.340181 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.340148    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-kube-api-access-xl6cp" pod "d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" (UID: "d1c1a136-bb43-4e4d-b47b-3d7c2af690ca"). InnerVolumeSpecName "kube-api-access-xl6cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:47:57.369123 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.369105    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:47:57.371118 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.371102    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 13:47:57.435493 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.435464    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-workload-certs\") pod \"d43b8d98-489b-4c47-8718-9d47509709b5\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") "
Nov 02 13:47:57.435625 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.435505    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-istio-data\") pod \"d43b8d98-489b-4c47-8718-9d47509709b5\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") "
Nov 02 13:47:57.435625 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.435545    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d43b8d98-489b-4c47-8718-9d47509709b5-kube-api-access-8ljjw\" (UniqueName: \"kubernetes.io/projected/d43b8d98-489b-4c47-8718-9d47509709b5-kube-api-access-8ljjw\") pod \"d43b8d98-489b-4c47-8718-9d47509709b5\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") "
Nov 02 13:47:57.435625 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.435580    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-istio-envoy\") pod \"d43b8d98-489b-4c47-8718-9d47509709b5\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") "
Nov 02 13:47:57.435625 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.435613    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-workload-socket\") pod \"d43b8d98-489b-4c47-8718-9d47509709b5\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") "
Nov 02 13:47:57.435867 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.435665    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-credential-socket\") pod \"d43b8d98-489b-4c47-8718-9d47509709b5\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") "
Nov 02 13:47:57.435867 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.435695    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d43b8d98-489b-4c47-8718-9d47509709b5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d43b8d98-489b-4c47-8718-9d47509709b5-istiod-ca-cert\") pod \"d43b8d98-489b-4c47-8718-9d47509709b5\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") "
Nov 02 13:47:57.435867 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.435730    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d43b8d98-489b-4c47-8718-9d47509709b5-istio-token\" (UniqueName: \"kubernetes.io/projected/d43b8d98-489b-4c47-8718-9d47509709b5-istio-token\") pod \"d43b8d98-489b-4c47-8718-9d47509709b5\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") "
Nov 02 13:47:57.435867 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.435762    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-workload-certs" pod "d43b8d98-489b-4c47-8718-9d47509709b5" (UID: "d43b8d98-489b-4c47-8718-9d47509709b5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.435867 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.435795    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d43b8d98-489b-4c47-8718-9d47509709b5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d43b8d98-489b-4c47-8718-9d47509709b5-istio-podinfo\") pod \"d43b8d98-489b-4c47-8718-9d47509709b5\" (UID: \"d43b8d98-489b-4c47-8718-9d47509709b5\") "
Nov 02 13:47:57.436127 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.435974    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-istio-data" pod "d43b8d98-489b-4c47-8718-9d47509709b5" (UID: "d43b8d98-489b-4c47-8718-9d47509709b5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.436127 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.436015    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-workload-socket" pod "d43b8d98-489b-4c47-8718-9d47509709b5" (UID: "d43b8d98-489b-4c47-8718-9d47509709b5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.436242 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.436149    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.436242 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.436159    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d43b8d98-489b-4c47-8718-9d47509709b5-istiod-ca-cert" pod "d43b8d98-489b-4c47-8718-9d47509709b5" (UID: "d43b8d98-489b-4c47-8718-9d47509709b5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:47:57.436242 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.436170    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.436242 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.436196    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.436242 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.436216    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.436242 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.436232    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xl6cp\" (UniqueName: \"kubernetes.io/projected/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-kube-api-access-xl6cp\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.436527 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.436249    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.436527 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.436265    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.436527 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.436279    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.436527 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.436292    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.436527 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.436337    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.436527 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.436477    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-credential-socket" pod "d43b8d98-489b-4c47-8718-9d47509709b5" (UID: "d43b8d98-489b-4c47-8718-9d47509709b5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.438748 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.438633    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d43b8d98-489b-4c47-8718-9d47509709b5-istio-token" pod "d43b8d98-489b-4c47-8718-9d47509709b5" (UID: "d43b8d98-489b-4c47-8718-9d47509709b5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:47:57.438748 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.438680    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d43b8d98-489b-4c47-8718-9d47509709b5-istio-podinfo" pod "d43b8d98-489b-4c47-8718-9d47509709b5" (UID: "d43b8d98-489b-4c47-8718-9d47509709b5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 13:47:57.438748 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.438689    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-istio-envoy" pod "d43b8d98-489b-4c47-8718-9d47509709b5" (UID: "d43b8d98-489b-4c47-8718-9d47509709b5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.438748 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.438718    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d43b8d98-489b-4c47-8718-9d47509709b5-kube-api-access-8ljjw" pod "d43b8d98-489b-4c47-8718-9d47509709b5" (UID: "d43b8d98-489b-4c47-8718-9d47509709b5"). InnerVolumeSpecName "kube-api-access-8ljjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:47:57.538009 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.537926    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-workload-socket\") pod \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") "
Nov 02 13:47:57.538009 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.537980    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-workload-certs\") pod \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") "
Nov 02 13:47:57.538190 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538017    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e8b30edd-8440-4c10-9c73-678c72c86222-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8b30edd-8440-4c10-9c73-678c72c86222-istio-podinfo\") pod \"e8b30edd-8440-4c10-9c73-678c72c86222\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") "
Nov 02 13:47:57.538190 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538052    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-credential-socket\") pod \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") "
Nov 02 13:47:57.538190 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538094    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-envoy\") pod \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") "
Nov 02 13:47:57.538190 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538159    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-credential-socket\") pod \"e8b30edd-8440-4c10-9c73-678c72c86222\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") "
Nov 02 13:47:57.538372 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538194    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8b30edd-8440-4c10-9c73-678c72c86222-kube-api-access-w7tlf\" (UniqueName: \"kubernetes.io/projected/e8b30edd-8440-4c10-9c73-678c72c86222-kube-api-access-w7tlf\") pod \"e8b30edd-8440-4c10-9c73-678c72c86222\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") "
Nov 02 13:47:57.538372 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538225    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-istio-data\") pod \"e8b30edd-8440-4c10-9c73-678c72c86222\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") "
Nov 02 13:47:57.538372 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538258    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8b30edd-8440-4c10-9c73-678c72c86222-istio-token\" (UniqueName: \"kubernetes.io/projected/e8b30edd-8440-4c10-9c73-678c72c86222-istio-token\") pod \"e8b30edd-8440-4c10-9c73-678c72c86222\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") "
Nov 02 13:47:57.538372 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538286    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-data\") pod \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") "
Nov 02 13:47:57.538372 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538331    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eea11872-8c21-4997-ab19-c9b81dbcf4fa-kube-api-access-scb8t\" (UniqueName: \"kubernetes.io/projected/eea11872-8c21-4997-ab19-c9b81dbcf4fa-kube-api-access-scb8t\") pod \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") "
Nov 02 13:47:57.538372 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538369    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istiod-ca-cert\") pod \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") "
Nov 02 13:47:57.538676 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538425    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-token\" (UniqueName: \"kubernetes.io/projected/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-token\") pod \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") "
Nov 02 13:47:57.538676 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538459    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e8b30edd-8440-4c10-9c73-678c72c86222-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8b30edd-8440-4c10-9c73-678c72c86222-istiod-ca-cert\") pod \"e8b30edd-8440-4c10-9c73-678c72c86222\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") "
Nov 02 13:47:57.538676 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538497    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-workload-socket\") pod \"e8b30edd-8440-4c10-9c73-678c72c86222\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") "
Nov 02 13:47:57.538676 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538530    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-workload-certs\") pod \"e8b30edd-8440-4c10-9c73-678c72c86222\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") "
Nov 02 13:47:57.538676 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538564    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-istio-envoy\") pod \"e8b30edd-8440-4c10-9c73-678c72c86222\" (UID: \"e8b30edd-8440-4c10-9c73-678c72c86222\") "
Nov 02 13:47:57.538676 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538610    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-podinfo\") pod \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\" (UID: \"eea11872-8c21-4997-ab19-c9b81dbcf4fa\") "
Nov 02 13:47:57.538676 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538609    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-workload-certs" pod "eea11872-8c21-4997-ab19-c9b81dbcf4fa" (UID: "eea11872-8c21-4997-ab19-c9b81dbcf4fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.538676 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538614    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-workload-socket" pod "eea11872-8c21-4997-ab19-c9b81dbcf4fa" (UID: "eea11872-8c21-4997-ab19-c9b81dbcf4fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.539103 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538938    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.539103 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538959    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d43b8d98-489b-4c47-8718-9d47509709b5-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.539103 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538973    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d43b8d98-489b-4c47-8718-9d47509709b5-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.539103 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.538988    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d43b8d98-489b-4c47-8718-9d47509709b5-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.539103 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.539003    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.539103 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.539017    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.539103 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.539030    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ljjw\" (UniqueName: \"kubernetes.io/projected/d43b8d98-489b-4c47-8718-9d47509709b5-kube-api-access-8ljjw\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.539103 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.539045    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d43b8d98-489b-4c47-8718-9d47509709b5-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.539602 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.539535    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-workload-certs" pod "e8b30edd-8440-4c10-9c73-678c72c86222" (UID: "e8b30edd-8440-4c10-9c73-678c72c86222"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.539976 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.539935    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-data" pod "eea11872-8c21-4997-ab19-c9b81dbcf4fa" (UID: "eea11872-8c21-4997-ab19-c9b81dbcf4fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.539976 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.539952    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-credential-socket" pod "e8b30edd-8440-4c10-9c73-678c72c86222" (UID: "e8b30edd-8440-4c10-9c73-678c72c86222"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.540270 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.540235    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-istio-data" pod "e8b30edd-8440-4c10-9c73-678c72c86222" (UID: "e8b30edd-8440-4c10-9c73-678c72c86222"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.540374 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.540309    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-credential-socket" pod "eea11872-8c21-4997-ab19-c9b81dbcf4fa" (UID: "eea11872-8c21-4997-ab19-c9b81dbcf4fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.540682 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.540659    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istiod-ca-cert" pod "eea11872-8c21-4997-ab19-c9b81dbcf4fa" (UID: "eea11872-8c21-4997-ab19-c9b81dbcf4fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:47:57.540970 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.540946    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8b30edd-8440-4c10-9c73-678c72c86222-istiod-ca-cert" pod "e8b30edd-8440-4c10-9c73-678c72c86222" (UID: "e8b30edd-8440-4c10-9c73-678c72c86222"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:47:57.541126 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.541052    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-workload-socket" pod "e8b30edd-8440-4c10-9c73-678c72c86222" (UID: "e8b30edd-8440-4c10-9c73-678c72c86222"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.546115 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.543363    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eea11872-8c21-4997-ab19-c9b81dbcf4fa-kube-api-access-scb8t" pod "eea11872-8c21-4997-ab19-c9b81dbcf4fa" (UID: "eea11872-8c21-4997-ab19-c9b81dbcf4fa"). InnerVolumeSpecName "kube-api-access-scb8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:47:57.546253 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.543406    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e8b30edd-8440-4c10-9c73-678c72c86222-istio-podinfo" pod "e8b30edd-8440-4c10-9c73-678c72c86222" (UID: "e8b30edd-8440-4c10-9c73-678c72c86222"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 13:47:57.546253 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.544038    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-envoy" pod "eea11872-8c21-4997-ab19-c9b81dbcf4fa" (UID: "eea11872-8c21-4997-ab19-c9b81dbcf4fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.546649 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.546625    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-podinfo" pod "eea11872-8c21-4997-ab19-c9b81dbcf4fa" (UID: "eea11872-8c21-4997-ab19-c9b81dbcf4fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 02 13:47:57.546792 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.546647    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-istio-envoy" pod "e8b30edd-8440-4c10-9c73-678c72c86222" (UID: "e8b30edd-8440-4c10-9c73-678c72c86222"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 02 13:47:57.546881 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.546800    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8b30edd-8440-4c10-9c73-678c72c86222-istio-token" pod "e8b30edd-8440-4c10-9c73-678c72c86222" (UID: "e8b30edd-8440-4c10-9c73-678c72c86222"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:47:57.547247 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.547221    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-token" pod "eea11872-8c21-4997-ab19-c9b81dbcf4fa" (UID: "eea11872-8c21-4997-ab19-c9b81dbcf4fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:47:57.548515 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.548488    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8b30edd-8440-4c10-9c73-678c72c86222-kube-api-access-w7tlf" pod "e8b30edd-8440-4c10-9c73-678c72c86222" (UID: "e8b30edd-8440-4c10-9c73-678c72c86222"). InnerVolumeSpecName "kube-api-access-w7tlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:47:57.639833 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.639800    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8b30edd-8440-4c10-9c73-678c72c86222-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.639833 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.639832    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.640018 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.639847    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.640018 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.639861    4942 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-credential-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.640018 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.639875    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7tlf\" (UniqueName: \"kubernetes.io/projected/e8b30edd-8440-4c10-9c73-678c72c86222-kube-api-access-w7tlf\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.640018 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.639889    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.640018 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.639902    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8b30edd-8440-4c10-9c73-678c72c86222-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.640018 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.639914    4942 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-data\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.640018 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.639928    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scb8t\" (UniqueName: \"kubernetes.io/projected/eea11872-8c21-4997-ab19-c9b81dbcf4fa-kube-api-access-scb8t\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.640018 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.639941    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.640018 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.639955    4942 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-token\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.640018 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.639968    4942 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8b30edd-8440-4c10-9c73-678c72c86222-istiod-ca-cert\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.640018 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.639981    4942 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-workload-socket\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.640018 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.639995    4942 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-workload-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.640018 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.640010    4942 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8b30edd-8440-4c10-9c73-678c72c86222-istio-envoy\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:57.640462 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:57.640024    4942 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eea11872-8c21-4997-ab19-c9b81dbcf4fa-istio-podinfo\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:47:58.030263 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.030225    4942 generic.go:356] "Generic (PLEG): container finished" podID="d43b8d98-489b-4c47-8718-9d47509709b5" containerID="ad98961077edd5d62663d02aedda2cf80ea5c9da68de1d2709c4241b18a8a5da" exitCode=0
Nov 02 13:47:58.030420 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.030276    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"
Nov 02 13:47:58.030420 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.030328    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn" event={"ID":"d43b8d98-489b-4c47-8718-9d47509709b5","Type":"ContainerDied","Data":"ad98961077edd5d62663d02aedda2cf80ea5c9da68de1d2709c4241b18a8a5da"}
Nov 02 13:47:58.030420 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.030393    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn" event={"ID":"d43b8d98-489b-4c47-8718-9d47509709b5","Type":"ContainerDied","Data":"4eac5edc61f5a98c653baf372415aa8ca4aab3d32381526501bb9c4703f3a028"}
Nov 02 13:47:58.030610 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.030423    4942 scope.go:122] "RemoveContainer" containerID="ad98961077edd5d62663d02aedda2cf80ea5c9da68de1d2709c4241b18a8a5da"
Nov 02 13:47:58.032011 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.031974    4942 generic.go:356] "Generic (PLEG): container finished" podID="e8b30edd-8440-4c10-9c73-678c72c86222" containerID="7c7ca367d19d30279ac69503d1406d6f31429a22e4577d425f67f71ac45f4cba" exitCode=0
Nov 02 13:47:58.032126 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.032019    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"
Nov 02 13:47:58.032126 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.032060    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" event={"ID":"e8b30edd-8440-4c10-9c73-678c72c86222","Type":"ContainerDied","Data":"7c7ca367d19d30279ac69503d1406d6f31429a22e4577d425f67f71ac45f4cba"}
Nov 02 13:47:58.032126 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.032113    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9" event={"ID":"e8b30edd-8440-4c10-9c73-678c72c86222","Type":"ContainerDied","Data":"9086aa4458ef5b09622cb16e7ae5ffb978300187bc976d2b56fa4b4ec3400cd4"}
Nov 02 13:47:58.033671 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.033647    4942 generic.go:356] "Generic (PLEG): container finished" podID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerID="815996b2c876bde0ef3067780939afb791d433f7c977d5dbbf3ac039faf10826" exitCode=0
Nov 02 13:47:58.033836 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.033687    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"
Nov 02 13:47:58.033836 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.033714    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" event={"ID":"eea11872-8c21-4997-ab19-c9b81dbcf4fa","Type":"ContainerDied","Data":"815996b2c876bde0ef3067780939afb791d433f7c977d5dbbf3ac039faf10826"}
Nov 02 13:47:58.033836 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.033766    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2" event={"ID":"eea11872-8c21-4997-ab19-c9b81dbcf4fa","Type":"ContainerDied","Data":"0021bc471661780e36f9faecf35587ff9f7d26dca49105d151b84ad768d090ca"}
Nov 02 13:47:58.035605 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.035582    4942 generic.go:356] "Generic (PLEG): container finished" podID="d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" containerID="cbd95db86fefff5631431a68c4a1902eaac0b4a6cf4f5bca717f3cfabefe157a" exitCode=0
Nov 02 13:47:58.035691 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.035622    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"
Nov 02 13:47:58.035691 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.035627    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs" event={"ID":"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca","Type":"ContainerDied","Data":"cbd95db86fefff5631431a68c4a1902eaac0b4a6cf4f5bca717f3cfabefe157a"}
Nov 02 13:47:58.035691 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.035669    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs" event={"ID":"d1c1a136-bb43-4e4d-b47b-3d7c2af690ca","Type":"ContainerDied","Data":"2a11d2d0bb418bb36e7b87a594b6994b006ebcac89b5eddf4d6d189a55df748b"}
Nov 02 13:47:58.040969 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.040944    4942 scope.go:122] "RemoveContainer" containerID="ad98961077edd5d62663d02aedda2cf80ea5c9da68de1d2709c4241b18a8a5da"
Nov 02 13:47:58.041220 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:47:58.041201    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad98961077edd5d62663d02aedda2cf80ea5c9da68de1d2709c4241b18a8a5da\": container with ID starting with ad98961077edd5d62663d02aedda2cf80ea5c9da68de1d2709c4241b18a8a5da not found: ID does not exist" containerID="ad98961077edd5d62663d02aedda2cf80ea5c9da68de1d2709c4241b18a8a5da"
Nov 02 13:47:58.041282 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.041229    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad98961077edd5d62663d02aedda2cf80ea5c9da68de1d2709c4241b18a8a5da"} err="failed to get container status \"ad98961077edd5d62663d02aedda2cf80ea5c9da68de1d2709c4241b18a8a5da\": rpc error: code = NotFound desc = could not find container \"ad98961077edd5d62663d02aedda2cf80ea5c9da68de1d2709c4241b18a8a5da\": container with ID starting with ad98961077edd5d62663d02aedda2cf80ea5c9da68de1d2709c4241b18a8a5da not found: ID does not exist"
Nov 02 13:47:58.041282 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.041245    4942 scope.go:122] "RemoveContainer" containerID="7c7ca367d19d30279ac69503d1406d6f31429a22e4577d425f67f71ac45f4cba"
Nov 02 13:47:58.049980 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.049964    4942 scope.go:122] "RemoveContainer" containerID="7c7ca367d19d30279ac69503d1406d6f31429a22e4577d425f67f71ac45f4cba"
Nov 02 13:47:58.050226 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:47:58.050208    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c7ca367d19d30279ac69503d1406d6f31429a22e4577d425f67f71ac45f4cba\": container with ID starting with 7c7ca367d19d30279ac69503d1406d6f31429a22e4577d425f67f71ac45f4cba not found: ID does not exist" containerID="7c7ca367d19d30279ac69503d1406d6f31429a22e4577d425f67f71ac45f4cba"
Nov 02 13:47:58.050277 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.050233    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c7ca367d19d30279ac69503d1406d6f31429a22e4577d425f67f71ac45f4cba"} err="failed to get container status \"7c7ca367d19d30279ac69503d1406d6f31429a22e4577d425f67f71ac45f4cba\": rpc error: code = NotFound desc = could not find container \"7c7ca367d19d30279ac69503d1406d6f31429a22e4577d425f67f71ac45f4cba\": container with ID starting with 7c7ca367d19d30279ac69503d1406d6f31429a22e4577d425f67f71ac45f4cba not found: ID does not exist"
Nov 02 13:47:58.050277 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.050252    4942 scope.go:122] "RemoveContainer" containerID="815996b2c876bde0ef3067780939afb791d433f7c977d5dbbf3ac039faf10826"
Nov 02 13:47:58.057980 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.057960    4942 scope.go:122] "RemoveContainer" containerID="815996b2c876bde0ef3067780939afb791d433f7c977d5dbbf3ac039faf10826"
Nov 02 13:47:58.058210 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:47:58.058191    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"815996b2c876bde0ef3067780939afb791d433f7c977d5dbbf3ac039faf10826\": container with ID starting with 815996b2c876bde0ef3067780939afb791d433f7c977d5dbbf3ac039faf10826 not found: ID does not exist" containerID="815996b2c876bde0ef3067780939afb791d433f7c977d5dbbf3ac039faf10826"
Nov 02 13:47:58.058266 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.058214    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"815996b2c876bde0ef3067780939afb791d433f7c977d5dbbf3ac039faf10826"} err="failed to get container status \"815996b2c876bde0ef3067780939afb791d433f7c977d5dbbf3ac039faf10826\": rpc error: code = NotFound desc = could not find container \"815996b2c876bde0ef3067780939afb791d433f7c977d5dbbf3ac039faf10826\": container with ID starting with 815996b2c876bde0ef3067780939afb791d433f7c977d5dbbf3ac039faf10826 not found: ID does not exist"
Nov 02 13:47:58.058266 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.058231    4942 scope.go:122] "RemoveContainer" containerID="cbd95db86fefff5631431a68c4a1902eaac0b4a6cf4f5bca717f3cfabefe157a"
Nov 02 13:47:58.065684 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.065665    4942 scope.go:122] "RemoveContainer" containerID="cbd95db86fefff5631431a68c4a1902eaac0b4a6cf4f5bca717f3cfabefe157a"
Nov 02 13:47:58.065941 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:47:58.065923    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbd95db86fefff5631431a68c4a1902eaac0b4a6cf4f5bca717f3cfabefe157a\": container with ID starting with cbd95db86fefff5631431a68c4a1902eaac0b4a6cf4f5bca717f3cfabefe157a not found: ID does not exist" containerID="cbd95db86fefff5631431a68c4a1902eaac0b4a6cf4f5bca717f3cfabefe157a"
Nov 02 13:47:58.065999 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.065947    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbd95db86fefff5631431a68c4a1902eaac0b4a6cf4f5bca717f3cfabefe157a"} err="failed to get container status \"cbd95db86fefff5631431a68c4a1902eaac0b4a6cf4f5bca717f3cfabefe157a\": rpc error: code = NotFound desc = could not find container \"cbd95db86fefff5631431a68c4a1902eaac0b4a6cf4f5bca717f3cfabefe157a\": container with ID starting with cbd95db86fefff5631431a68c4a1902eaac0b4a6cf4f5bca717f3cfabefe157a not found: ID does not exist"
Nov 02 13:47:58.194789 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.194757    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"]
Nov 02 13:47:58.244585 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.244516    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-qzbzn"]
Nov 02 13:47:58.312339 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.312309    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"]
Nov 02 13:47:58.318920 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.318895    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-7hsh2"]
Nov 02 13:47:58.427733 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.427703    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"]
Nov 02 13:47:58.432371 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.432347    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-wkxz9"]
Nov 02 13:47:58.544507 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.544010    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"]
Nov 02 13:47:58.549426 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:58.549398    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-pjqbs"]
Nov 02 13:47:59.542007 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:59.541974    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" path="/var/lib/kubelet/pods/d1c1a136-bb43-4e4d-b47b-3d7c2af690ca/volumes"
Nov 02 13:47:59.542435 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:59.542420    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d43b8d98-489b-4c47-8718-9d47509709b5" path="/var/lib/kubelet/pods/d43b8d98-489b-4c47-8718-9d47509709b5/volumes"
Nov 02 13:47:59.542820 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:59.542808    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" path="/var/lib/kubelet/pods/e8b30edd-8440-4c10-9c73-678c72c86222/volumes"
Nov 02 13:47:59.543521 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:47:59.543507    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" path="/var/lib/kubelet/pods/eea11872-8c21-4997-ab19-c9b81dbcf4fa/volumes"
Nov 02 13:48:00.982403 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:48:00.982371    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:48:30.982801 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:48:30.982765    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:49:00.982076 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:49:00.982039    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:49:30.981417 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:49:30.981342    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:50:00.982450 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:50:00.982413    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:50:08.031515 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.031480    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"]
Nov 02 13:50:08.031912 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.031859    4942 cpu_manager.go:414] "containerMap: removing container" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy"
Nov 02 13:50:08.031912 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.031871    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy"
Nov 02 13:50:08.031912 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.031882    4942 cpu_manager.go:414] "containerMap: removing container" podUID="d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" containerName="istio-proxy"
Nov 02 13:50:08.031912 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.031887    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" containerName="istio-proxy"
Nov 02 13:50:08.031912 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.031899    4942 cpu_manager.go:414] "containerMap: removing container" podUID="55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3" containerName="maas-api"
Nov 02 13:50:08.031912 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.031904    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3" containerName="maas-api"
Nov 02 13:50:08.031912 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.031913    4942 cpu_manager.go:414] "containerMap: removing container" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" containerName="istio-proxy"
Nov 02 13:50:08.032132 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.031919    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" containerName="istio-proxy"
Nov 02 13:50:08.032132 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.031928    4942 cpu_manager.go:414] "containerMap: removing container" podUID="d43b8d98-489b-4c47-8718-9d47509709b5" containerName="istio-proxy"
Nov 02 13:50:08.032132 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.031933    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d43b8d98-489b-4c47-8718-9d47509709b5" containerName="istio-proxy"
Nov 02 13:50:08.032132 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.031988    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="eea11872-8c21-4997-ab19-c9b81dbcf4fa" containerName="istio-proxy"
Nov 02 13:50:08.032132 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.031996    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1c1a136-bb43-4e4d-b47b-3d7c2af690ca" containerName="istio-proxy"
Nov 02 13:50:08.032132 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.032002    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="55bb3e61-cc4a-48ec-a5ad-c3d97233b9e3" containerName="maas-api"
Nov 02 13:50:08.032132 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.032009    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8b30edd-8440-4c10-9c73-678c72c86222" containerName="istio-proxy"
Nov 02 13:50:08.032132 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.032015    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="d43b8d98-489b-4c47-8718-9d47509709b5" containerName="istio-proxy"
Nov 02 13:50:08.035030 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.035011    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.104237 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.104214    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-c5ldd\""
Nov 02 13:50:08.158552 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.158526    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61489084-5c87-4d75-8ccd-ddeea606747a-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.158702 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.158565    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61489084-5c87-4d75-8ccd-ddeea606747a-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.158702 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.158594    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61489084-5c87-4d75-8ccd-ddeea606747a-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.158838 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.158732    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61489084-5c87-4d75-8ccd-ddeea606747a-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.158838 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.158776    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61489084-5c87-4d75-8ccd-ddeea606747a-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.158948 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.158854    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61489084-5c87-4d75-8ccd-ddeea606747a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.158948 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.158902    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61489084-5c87-4d75-8ccd-ddeea606747a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.158948 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.158929    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg78q\" (UniqueName: \"kubernetes.io/projected/61489084-5c87-4d75-8ccd-ddeea606747a-kube-api-access-hg78q\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.159116 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.159057    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61489084-5c87-4d75-8ccd-ddeea606747a-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.194207 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.194175    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"]
Nov 02 13:50:08.259968 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.259933    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61489084-5c87-4d75-8ccd-ddeea606747a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.260119 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.259987    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61489084-5c87-4d75-8ccd-ddeea606747a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.260119 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.260013    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg78q\" (UniqueName: \"kubernetes.io/projected/61489084-5c87-4d75-8ccd-ddeea606747a-kube-api-access-hg78q\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.260119 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.260058    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61489084-5c87-4d75-8ccd-ddeea606747a-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.260323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.260139    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61489084-5c87-4d75-8ccd-ddeea606747a-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.260323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.260168    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61489084-5c87-4d75-8ccd-ddeea606747a-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.260323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.260193    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61489084-5c87-4d75-8ccd-ddeea606747a-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.260323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.260237    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61489084-5c87-4d75-8ccd-ddeea606747a-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.260323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.260277    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61489084-5c87-4d75-8ccd-ddeea606747a-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.260577 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.260543    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61489084-5c87-4d75-8ccd-ddeea606747a-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.260668 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.260643    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61489084-5c87-4d75-8ccd-ddeea606747a-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.260730 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.260709    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61489084-5c87-4d75-8ccd-ddeea606747a-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.260791 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.260771    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61489084-5c87-4d75-8ccd-ddeea606747a-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.260845 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.260784    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61489084-5c87-4d75-8ccd-ddeea606747a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.262455 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.262429    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61489084-5c87-4d75-8ccd-ddeea606747a-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.262573 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.262556    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61489084-5c87-4d75-8ccd-ddeea606747a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.321123 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.321061    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61489084-5c87-4d75-8ccd-ddeea606747a-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.321221 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.321183    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg78q\" (UniqueName: \"kubernetes.io/projected/61489084-5c87-4d75-8ccd-ddeea606747a-kube-api-access-hg78q\") pod \"maas-default-gateway-openshift-default-6dcf757899-t2q27\" (UID: \"61489084-5c87-4d75-8ccd-ddeea606747a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.348883 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.348859    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:08.773108 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.773082    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"]
Nov 02 13:50:08.774833 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:50:08.774803    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61489084_5c87_4d75_8ccd_ddeea606747a.slice/crio-fbc65a2801eb67ef695aefb5ca16f2df5ddddb8615f769df3ecb2a1e64008bc5 WatchSource:0}: Error finding container fbc65a2801eb67ef695aefb5ca16f2df5ddddb8615f769df3ecb2a1e64008bc5: Status 404 returned error can't find the container with id fbc65a2801eb67ef695aefb5ca16f2df5ddddb8615f769df3ecb2a1e64008bc5
Nov 02 13:50:08.776896 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.776863    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:50:08.776972 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.776939    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:50:08.776972 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:08.776967    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:50:09.707322 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:09.707264    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27" event={"ID":"61489084-5c87-4d75-8ccd-ddeea606747a","Type":"ContainerStarted","Data":"3b18d6924e643edf7b0e9f46f348132c77976c6b390a935dfbb535db97f081ab"}
Nov 02 13:50:09.707748 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:09.707346    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27" event={"ID":"61489084-5c87-4d75-8ccd-ddeea606747a","Type":"ContainerStarted","Data":"fbc65a2801eb67ef695aefb5ca16f2df5ddddb8615f769df3ecb2a1e64008bc5"}
Nov 02 13:50:09.928145 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:09.928091    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27" podStartSLOduration=2.928077936 podStartE2EDuration="2.928077936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:50:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:50:09.878023411 +0000 UTC m=+6836.449373180" watchObservedRunningTime="2026-11-02 13:50:09.928077936 +0000 UTC m=+6836.499427693"
Nov 02 13:50:10.349464 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:10.349431    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:10.350786 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:10.350758    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-t2q27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.143:15021/healthz/ready\": dial tcp 10.132.0.143:15021: connect: connection refused" start-of-body=
Nov 02 13:50:10.350916 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:10.350810    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27" podUID="61489084-5c87-4d75-8ccd-ddeea606747a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.143:15021/healthz/ready\": dial tcp 10.132.0.143:15021: connect: connection refused"
Nov 02 13:50:11.349396 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:11.349361    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-t2q27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.143:15021/healthz/ready\": dial tcp 10.132.0.143:15021: connect: connection refused" start-of-body=
Nov 02 13:50:11.349773 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:11.349420    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27" podUID="61489084-5c87-4d75-8ccd-ddeea606747a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.143:15021/healthz/ready\": dial tcp 10.132.0.143:15021: connect: connection refused"
Nov 02 13:50:12.349534 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:12.349493    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-t2q27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.143:15021/healthz/ready\": dial tcp 10.132.0.143:15021: connect: connection refused" start-of-body=
Nov 02 13:50:12.349934 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:12.349550    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27" podUID="61489084-5c87-4d75-8ccd-ddeea606747a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.143:15021/healthz/ready\": dial tcp 10.132.0.143:15021: connect: connection refused"
Nov 02 13:50:13.349633 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:13.349591    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-t2q27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.143:15021/healthz/ready\": dial tcp 10.132.0.143:15021: connect: connection refused" start-of-body=
Nov 02 13:50:13.350135 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:13.349657    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27" podUID="61489084-5c87-4d75-8ccd-ddeea606747a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.143:15021/healthz/ready\": dial tcp 10.132.0.143:15021: connect: connection refused"
Nov 02 13:50:14.350223 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:14.350185    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-t2q27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.143:15021/healthz/ready\": dial tcp 10.132.0.143:15021: connect: connection refused" start-of-body=
Nov 02 13:50:14.350641 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:14.350251    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27" podUID="61489084-5c87-4d75-8ccd-ddeea606747a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.143:15021/healthz/ready\": dial tcp 10.132.0.143:15021: connect: connection refused"
Nov 02 13:50:15.349525 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:15.349479    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-t2q27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.143:15021/healthz/ready\": dial tcp 10.132.0.143:15021: connect: connection refused" start-of-body=
Nov 02 13:50:15.349766 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:15.349558    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27" podUID="61489084-5c87-4d75-8ccd-ddeea606747a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.143:15021/healthz/ready\": dial tcp 10.132.0.143:15021: connect: connection refused"
Nov 02 13:50:16.349281 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:16.349236    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-t2q27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.143:15021/healthz/ready\": dial tcp 10.132.0.143:15021: connect: connection refused" start-of-body=
Nov 02 13:50:16.349842 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:16.349290    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27" podUID="61489084-5c87-4d75-8ccd-ddeea606747a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.143:15021/healthz/ready\": dial tcp 10.132.0.143:15021: connect: connection refused"
Nov 02 13:50:18.138640 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:18.138605    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:18.139133 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:18.139075    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:18.140003 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:18.139977    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27"
Nov 02 13:50:23.156349 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.156317    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"]
Nov 02 13:50:23.159972 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.159952    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.197833 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.197808    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b55accad-1dd4-4a22-8f14-cddd8a0283af-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.197962 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.197862    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b55accad-1dd4-4a22-8f14-cddd8a0283af-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.197962 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.197928    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b55accad-1dd4-4a22-8f14-cddd8a0283af-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.198036 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.197995    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b55accad-1dd4-4a22-8f14-cddd8a0283af-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.198036 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.198026    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp87m\" (UniqueName: \"kubernetes.io/projected/b55accad-1dd4-4a22-8f14-cddd8a0283af-kube-api-access-zp87m\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.198124 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.198110    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b55accad-1dd4-4a22-8f14-cddd8a0283af-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.198166 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.198132    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b55accad-1dd4-4a22-8f14-cddd8a0283af-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.198166 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.198148    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b55accad-1dd4-4a22-8f14-cddd8a0283af-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.198260 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.198170    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b55accad-1dd4-4a22-8f14-cddd8a0283af-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.261611 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.261578    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"]
Nov 02 13:50:23.299586 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.299555    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b55accad-1dd4-4a22-8f14-cddd8a0283af-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.299736 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.299597    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b55accad-1dd4-4a22-8f14-cddd8a0283af-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.299736 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.299622    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b55accad-1dd4-4a22-8f14-cddd8a0283af-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.299736 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.299660    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp87m\" (UniqueName: \"kubernetes.io/projected/b55accad-1dd4-4a22-8f14-cddd8a0283af-kube-api-access-zp87m\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.299736 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.299713    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b55accad-1dd4-4a22-8f14-cddd8a0283af-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.299979 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.299742    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b55accad-1dd4-4a22-8f14-cddd8a0283af-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.299979 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.299763    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b55accad-1dd4-4a22-8f14-cddd8a0283af-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.299979 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.299794    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b55accad-1dd4-4a22-8f14-cddd8a0283af-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.299979 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.299854    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b55accad-1dd4-4a22-8f14-cddd8a0283af-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.300233 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.300204    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b55accad-1dd4-4a22-8f14-cddd8a0283af-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.300335 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.300252    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b55accad-1dd4-4a22-8f14-cddd8a0283af-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.300335 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.300269    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b55accad-1dd4-4a22-8f14-cddd8a0283af-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.300455 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.300377    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b55accad-1dd4-4a22-8f14-cddd8a0283af-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.300513 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.300459    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b55accad-1dd4-4a22-8f14-cddd8a0283af-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.302185 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.302157    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b55accad-1dd4-4a22-8f14-cddd8a0283af-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.302348 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.302331    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b55accad-1dd4-4a22-8f14-cddd8a0283af-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.360582 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.360554    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b55accad-1dd4-4a22-8f14-cddd8a0283af-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.360684 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.360674    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp87m\" (UniqueName: \"kubernetes.io/projected/b55accad-1dd4-4a22-8f14-cddd8a0283af-kube-api-access-zp87m\") pod \"maas-default-gateway-openshift-default-6dcf757899-z6h47\" (UID: \"b55accad-1dd4-4a22-8f14-cddd8a0283af\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.473682 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.473607    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:23.902120 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.902096    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"]
Nov 02 13:50:23.903116 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:50:23.903085    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb55accad_1dd4_4a22_8f14_cddd8a0283af.slice/crio-977821de9b30af9fddd8442e9cdab10fe617b988bedb0e3e876f56d602ab4591 WatchSource:0}: Error finding container 977821de9b30af9fddd8442e9cdab10fe617b988bedb0e3e876f56d602ab4591: Status 404 returned error can't find the container with id 977821de9b30af9fddd8442e9cdab10fe617b988bedb0e3e876f56d602ab4591
Nov 02 13:50:23.905402 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.905371    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:50:23.905481 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.905433    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:50:23.905481 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:23.905462    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:50:24.786411 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:24.786374    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47" event={"ID":"b55accad-1dd4-4a22-8f14-cddd8a0283af","Type":"ContainerStarted","Data":"c3544fe0a6485b5b8cba85727410ec7d76846e95555c0833a605faa5dee3b0ad"}
Nov 02 13:50:24.786825 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:24.786432    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47" event={"ID":"b55accad-1dd4-4a22-8f14-cddd8a0283af","Type":"ContainerStarted","Data":"977821de9b30af9fddd8442e9cdab10fe617b988bedb0e3e876f56d602ab4591"}
Nov 02 13:50:25.477471 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:25.477432    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:25.479063 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:25.479032    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-z6h47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Nov 02 13:50:25.479194 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:25.479080    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47" podUID="b55accad-1dd4-4a22-8f14-cddd8a0283af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Nov 02 13:50:26.477448 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:26.477405    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-z6h47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Nov 02 13:50:26.477877 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:26.477475    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47" podUID="b55accad-1dd4-4a22-8f14-cddd8a0283af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Nov 02 13:50:27.477777 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:27.477735    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-z6h47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Nov 02 13:50:27.478213 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:27.477805    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47" podUID="b55accad-1dd4-4a22-8f14-cddd8a0283af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Nov 02 13:50:28.477323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:28.476678    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-z6h47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Nov 02 13:50:28.477323 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:28.476743    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47" podUID="b55accad-1dd4-4a22-8f14-cddd8a0283af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Nov 02 13:50:29.477736 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:29.477690    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-z6h47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Nov 02 13:50:29.478146 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:29.477749    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47" podUID="b55accad-1dd4-4a22-8f14-cddd8a0283af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Nov 02 13:50:30.474310 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:30.474265    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-z6h47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Nov 02 13:50:30.474514 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:30.474334    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47" podUID="b55accad-1dd4-4a22-8f14-cddd8a0283af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Nov 02 13:50:30.983062 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:50:30.983028    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:50:31.473965 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:31.473930    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-z6h47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused" start-of-body=
Nov 02 13:50:31.474168 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:31.473988    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47" podUID="b55accad-1dd4-4a22-8f14-cddd8a0283af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": dial tcp 10.132.0.144:15021: connect: connection refused"
Nov 02 13:50:33.266292 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:33.266256    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:33.266830 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:33.266734    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:33.267617 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:33.267594    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47"
Nov 02 13:50:33.462988 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:50:33.462932    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47" podStartSLOduration=10.462919861 podStartE2EDuration="10.462919861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:50:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:50:24.969148745 +0000 UTC m=+6851.540498503" watchObservedRunningTime="2026-11-02 13:50:33.462919861 +0000 UTC m=+6860.034269618"
Nov 02 13:51:00.983174 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:51:00.983088    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:51:14.468270 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:14.468243    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:51:14.495215 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:14.495189    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:51:30.981862 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:51:30.981821    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:51:53.890773 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:53.890738    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"]
Nov 02 13:51:53.894161 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:53.894143    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:53.917850 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:53.917821    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:53.917975 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:53.917859    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:53.917975 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:53.917880    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:53.918060 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:53.918018    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:53.918060 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:53.918052    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:53.918142 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:53.918077    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbff9\" (UniqueName: \"kubernetes.io/projected/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-kube-api-access-cbff9\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:53.918196 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:53.918179    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:53.918234 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:53.918207    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:53.918279 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:53.918232    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.019183 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.019148    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.019401 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.019199    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.019401 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.019238    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbff9\" (UniqueName: \"kubernetes.io/projected/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-kube-api-access-cbff9\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.019401 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.019325    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.019600 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.019465    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.019600 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.019528    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.019600 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.019559    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.019600 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.019595    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.019815 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.019627    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.019923 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.019901    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-workload-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.019991 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.019937    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-istio-data\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.020053 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.019989    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-credential-socket\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.020113 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.020093    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-workload-certs\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.020174 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.020132    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.021622 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.021600    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-istio-envoy\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.021961 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.021942    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.079914 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.079880    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-istio-token\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.080055 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.079995    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbff9\" (UniqueName: \"kubernetes.io/projected/4428344b-e3a5-4dd1-b784-6a8d1dec4a37-kube-api-access-cbff9\") pod \"maas-default-gateway-openshift-default-6dcf757899-547v7\" (UID: \"4428344b-e3a5-4dd1-b784-6a8d1dec4a37\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.100807 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.100777    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"]
Nov 02 13:51:54.206987 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.206905    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:54.632154 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.632128    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"]
Nov 02 13:51:54.634438 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:51:54.634400    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4428344b_e3a5_4dd1_b784_6a8d1dec4a37.slice/crio-6501307de158e70db5ce2bc83e3a78a0377a04e5d94ba93117aa057b89b9c949 WatchSource:0}: Error finding container 6501307de158e70db5ce2bc83e3a78a0377a04e5d94ba93117aa057b89b9c949: Status 404 returned error can't find the container with id 6501307de158e70db5ce2bc83e3a78a0377a04e5d94ba93117aa057b89b9c949
Nov 02 13:51:54.636657 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.636623    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:51:54.636732 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.636706    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:51:54.636771 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:54.636737    4942 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Nov 02 13:51:55.279188 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:55.279146    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7" event={"ID":"4428344b-e3a5-4dd1-b784-6a8d1dec4a37","Type":"ContainerStarted","Data":"622c7e7d5a7143db752fe0cedc7d721f4e2a9c6a213487d4a08da15cc61a0d77"}
Nov 02 13:51:55.279648 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:55.279208    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7" event={"ID":"4428344b-e3a5-4dd1-b784-6a8d1dec4a37","Type":"ContainerStarted","Data":"6501307de158e70db5ce2bc83e3a78a0377a04e5d94ba93117aa057b89b9c949"}
Nov 02 13:51:55.479366 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:55.479281    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7" podStartSLOduration=2.479262395 podStartE2EDuration="2.479262395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:51:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:51:55.47675443 +0000 UTC m=+6942.048104189" watchObservedRunningTime="2026-11-02 13:51:55.479262395 +0000 UTC m=+6942.050612209"
Nov 02 13:51:56.207262 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:56.207227    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:51:56.208767 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:56.208745    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-547v7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused" start-of-body=
Nov 02 13:51:56.208830 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:56.208791    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7" podUID="4428344b-e3a5-4dd1-b784-6a8d1dec4a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused"
Nov 02 13:51:57.207237 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:57.207197    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-547v7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused" start-of-body=
Nov 02 13:51:57.207656 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:57.207258    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7" podUID="4428344b-e3a5-4dd1-b784-6a8d1dec4a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused"
Nov 02 13:51:58.207681 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:58.207643    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-547v7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused" start-of-body=
Nov 02 13:51:58.208110 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:58.207702    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7" podUID="4428344b-e3a5-4dd1-b784-6a8d1dec4a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused"
Nov 02 13:51:59.207314 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:59.207265    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-547v7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused" start-of-body=
Nov 02 13:51:59.207510 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:51:59.207347    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7" podUID="4428344b-e3a5-4dd1-b784-6a8d1dec4a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused"
Nov 02 13:52:00.207591 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:52:00.207552    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-547v7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused" start-of-body=
Nov 02 13:52:00.208083 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:52:00.207617    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7" podUID="4428344b-e3a5-4dd1-b784-6a8d1dec4a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused"
Nov 02 13:52:00.982458 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:52:00.982417    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:52:01.207678 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:52:01.207645    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-547v7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused" start-of-body=
Nov 02 13:52:01.208071 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:52:01.207696    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7" podUID="4428344b-e3a5-4dd1-b784-6a8d1dec4a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused"
Nov 02 13:52:02.207843 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:52:02.207811    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-547v7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused" start-of-body=
Nov 02 13:52:02.208248 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:52:02.207861    4942 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7" podUID="4428344b-e3a5-4dd1-b784-6a8d1dec4a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused"
Nov 02 13:52:03.792941 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:52:03.792906    4942 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:52:03.793461 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:52:03.793236    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:52:03.794379 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:52:03.794355    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7"
Nov 02 13:52:30.983360 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:52:30.983265    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:53:00.982937 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:53:00.982900    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:53:12.260322 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:12.260265    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"]
Nov 02 13:53:12.264533 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:12.264505    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:53:12.282315 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:12.282271    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c79bx\" (UniqueName: \"kubernetes.io/projected/fd396f31-6170-4a8b-96ee-358fec42f95a-kube-api-access-c79bx\") pod \"maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4\" (UID: \"fd396f31-6170-4a8b-96ee-358fec42f95a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:53:12.282456 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:12.282395    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd396f31-6170-4a8b-96ee-358fec42f95a-openshift-service-ca\") pod \"maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4\" (UID: \"fd396f31-6170-4a8b-96ee-358fec42f95a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:53:12.282521 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:12.282455    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd396f31-6170-4a8b-96ee-358fec42f95a-maas-api-tls\") pod \"maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4\" (UID: \"fd396f31-6170-4a8b-96ee-358fec42f95a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:53:12.282521 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:12.282489    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd396f31-6170-4a8b-96ee-358fec42f95a-metrics-certs\") pod \"maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4\" (UID: \"fd396f31-6170-4a8b-96ee-358fec42f95a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:53:12.316542 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:12.316505    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-8869c0de-serving-cert\""
Nov 02 13:53:12.369604 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:12.369568    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"]
Nov 02 13:53:12.383692 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:12.383661    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c79bx\" (UniqueName: \"kubernetes.io/projected/fd396f31-6170-4a8b-96ee-358fec42f95a-kube-api-access-c79bx\") pod \"maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4\" (UID: \"fd396f31-6170-4a8b-96ee-358fec42f95a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:53:12.383852 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:12.383714    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd396f31-6170-4a8b-96ee-358fec42f95a-openshift-service-ca\") pod \"maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4\" (UID: \"fd396f31-6170-4a8b-96ee-358fec42f95a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:53:12.383852 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:12.383775    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd396f31-6170-4a8b-96ee-358fec42f95a-maas-api-tls\") pod \"maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4\" (UID: \"fd396f31-6170-4a8b-96ee-358fec42f95a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:53:12.383852 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:12.383819    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd396f31-6170-4a8b-96ee-358fec42f95a-metrics-certs\") pod \"maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4\" (UID: \"fd396f31-6170-4a8b-96ee-358fec42f95a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:53:12.383999 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:53:12.383894    4942 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-serving-cert: secret "maas-api-e2e-mt-8869c0de-serving-cert" not found
Nov 02 13:53:12.383999 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:53:12.383965    4942 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd396f31-6170-4a8b-96ee-358fec42f95a-maas-api-tls podName:fd396f31-6170-4a8b-96ee-358fec42f95a nodeName:}" failed. No retries permitted until 2026-11-02 13:53:12.883946318 +0000 UTC m=+7019.455296056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fd396f31-6170-4a8b-96ee-358fec42f95a-maas-api-tls") pod "maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4" (UID: "fd396f31-6170-4a8b-96ee-358fec42f95a") : secret "maas-api-e2e-mt-8869c0de-serving-cert" not found
Nov 02 13:53:12.384468 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:12.384449    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd396f31-6170-4a8b-96ee-358fec42f95a-openshift-service-ca\") pod \"maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4\" (UID: \"fd396f31-6170-4a8b-96ee-358fec42f95a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:53:12.386196 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:12.386176    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd396f31-6170-4a8b-96ee-358fec42f95a-metrics-certs\") pod \"maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4\" (UID: \"fd396f31-6170-4a8b-96ee-358fec42f95a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:53:12.447282 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:12.447248    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c79bx\" (UniqueName: \"kubernetes.io/projected/fd396f31-6170-4a8b-96ee-358fec42f95a-kube-api-access-c79bx\") pod \"maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4\" (UID: \"fd396f31-6170-4a8b-96ee-358fec42f95a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:53:12.889682 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:12.889647    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd396f31-6170-4a8b-96ee-358fec42f95a-maas-api-tls\") pod \"maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4\" (UID: \"fd396f31-6170-4a8b-96ee-358fec42f95a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:53:12.892006 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:12.891980    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd396f31-6170-4a8b-96ee-358fec42f95a-maas-api-tls\") pod \"maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4\" (UID: \"fd396f31-6170-4a8b-96ee-358fec42f95a\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:53:13.177293 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:13.177215    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:53:13.551289 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:13.551267    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"]
Nov 02 13:53:13.552953 ip-10-0-135-24 kubenswrapper[4942]: W1102 13:53:13.552913    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd396f31_6170_4a8b_96ee_358fec42f95a.slice/crio-ef7c94f561cdf3c27fab1850745c353e20edfecdc2655735a0da5939c479e1b0 WatchSource:0}: Error finding container ef7c94f561cdf3c27fab1850745c353e20edfecdc2655735a0da5939c479e1b0: Status 404 returned error can't find the container with id ef7c94f561cdf3c27fab1850745c353e20edfecdc2655735a0da5939c479e1b0
Nov 02 13:53:13.554260 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:13.554239    4942 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 13:53:13.689356 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:13.689322    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4" event={"ID":"fd396f31-6170-4a8b-96ee-358fec42f95a","Type":"ContainerStarted","Data":"ef7c94f561cdf3c27fab1850745c353e20edfecdc2655735a0da5939c479e1b0"}
Nov 02 13:53:14.697592 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:14.697555    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4" event={"ID":"fd396f31-6170-4a8b-96ee-358fec42f95a","Type":"ContainerStarted","Data":"73d05389b5bd29e43f1b15e9e068ecb129ed42a97bc248ecf6f5b875c8b7cdd6"}
Nov 02 13:53:14.698018 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:14.697716    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:53:14.834928 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:14.834833    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4" podStartSLOduration=1.837566184 podStartE2EDuration="2.834819319s" totalImagesPullingTime="997.253135ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 13:53:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 13:53:14.832810791 +0000 UTC m=+7021.404160543" watchObservedRunningTime="2026-11-02 13:53:14.834819319 +0000 UTC m=+7021.406169076"
Nov 02 13:53:20.707995 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:53:20.707961    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:53:30.982614 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:53:30.982576    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:54:00.983065 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:54:00.982991    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:54:30.982641 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:54:30.982604    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:55:00.982547 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:55:00.982509    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:55:10.943412 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:10.943373    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"]
Nov 02 13:55:10.943890 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:10.943568    4942 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4" podUID="fd396f31-6170-4a8b-96ee-358fec42f95a" containerName="maas-api" containerID="cri-o://73d05389b5bd29e43f1b15e9e068ecb129ed42a97bc248ecf6f5b875c8b7cdd6" gracePeriod=30
Nov 02 13:55:11.217669 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.217641    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:55:11.290446 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.290417    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd396f31-6170-4a8b-96ee-358fec42f95a-kube-api-access-c79bx\" (UniqueName: \"kubernetes.io/projected/fd396f31-6170-4a8b-96ee-358fec42f95a-kube-api-access-c79bx\") pod \"fd396f31-6170-4a8b-96ee-358fec42f95a\" (UID: \"fd396f31-6170-4a8b-96ee-358fec42f95a\") "
Nov 02 13:55:11.290616 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.290476    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fd396f31-6170-4a8b-96ee-358fec42f95a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd396f31-6170-4a8b-96ee-358fec42f95a-maas-api-tls\") pod \"fd396f31-6170-4a8b-96ee-358fec42f95a\" (UID: \"fd396f31-6170-4a8b-96ee-358fec42f95a\") "
Nov 02 13:55:11.290616 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.290527    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fd396f31-6170-4a8b-96ee-358fec42f95a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd396f31-6170-4a8b-96ee-358fec42f95a-metrics-certs\") pod \"fd396f31-6170-4a8b-96ee-358fec42f95a\" (UID: \"fd396f31-6170-4a8b-96ee-358fec42f95a\") "
Nov 02 13:55:11.290616 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.290571    4942 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd396f31-6170-4a8b-96ee-358fec42f95a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd396f31-6170-4a8b-96ee-358fec42f95a-openshift-service-ca\") pod \"fd396f31-6170-4a8b-96ee-358fec42f95a\" (UID: \"fd396f31-6170-4a8b-96ee-358fec42f95a\") "
Nov 02 13:55:11.291035 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.291007    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd396f31-6170-4a8b-96ee-358fec42f95a-openshift-service-ca" pod "fd396f31-6170-4a8b-96ee-358fec42f95a" (UID: "fd396f31-6170-4a8b-96ee-358fec42f95a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 02 13:55:11.292783 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.292749    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd396f31-6170-4a8b-96ee-358fec42f95a-kube-api-access-c79bx" pod "fd396f31-6170-4a8b-96ee-358fec42f95a" (UID: "fd396f31-6170-4a8b-96ee-358fec42f95a"). InnerVolumeSpecName "kube-api-access-c79bx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 02 13:55:11.292783 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.292754    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd396f31-6170-4a8b-96ee-358fec42f95a-maas-api-tls" pod "fd396f31-6170-4a8b-96ee-358fec42f95a" (UID: "fd396f31-6170-4a8b-96ee-358fec42f95a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:55:11.292985 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.292793    4942 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd396f31-6170-4a8b-96ee-358fec42f95a-metrics-certs" pod "fd396f31-6170-4a8b-96ee-358fec42f95a" (UID: "fd396f31-6170-4a8b-96ee-358fec42f95a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 02 13:55:11.320565 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.320536    4942 generic.go:356] "Generic (PLEG): container finished" podID="fd396f31-6170-4a8b-96ee-358fec42f95a" containerID="73d05389b5bd29e43f1b15e9e068ecb129ed42a97bc248ecf6f5b875c8b7cdd6" exitCode=0
Nov 02 13:55:11.320674 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.320582    4942 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"
Nov 02 13:55:11.320674 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.320613    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4" event={"ID":"fd396f31-6170-4a8b-96ee-358fec42f95a","Type":"ContainerDied","Data":"73d05389b5bd29e43f1b15e9e068ecb129ed42a97bc248ecf6f5b875c8b7cdd6"}
Nov 02 13:55:11.320747 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.320697    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4" event={"ID":"fd396f31-6170-4a8b-96ee-358fec42f95a","Type":"ContainerDied","Data":"ef7c94f561cdf3c27fab1850745c353e20edfecdc2655735a0da5939c479e1b0"}
Nov 02 13:55:11.320747 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.320725    4942 scope.go:122] "RemoveContainer" containerID="73d05389b5bd29e43f1b15e9e068ecb129ed42a97bc248ecf6f5b875c8b7cdd6"
Nov 02 13:55:11.332623 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.332601    4942 scope.go:122] "RemoveContainer" containerID="73d05389b5bd29e43f1b15e9e068ecb129ed42a97bc248ecf6f5b875c8b7cdd6"
Nov 02 13:55:11.333070 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:55:11.333040    4942 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73d05389b5bd29e43f1b15e9e068ecb129ed42a97bc248ecf6f5b875c8b7cdd6\": container with ID starting with 73d05389b5bd29e43f1b15e9e068ecb129ed42a97bc248ecf6f5b875c8b7cdd6 not found: ID does not exist" containerID="73d05389b5bd29e43f1b15e9e068ecb129ed42a97bc248ecf6f5b875c8b7cdd6"
Nov 02 13:55:11.333200 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.333070    4942 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73d05389b5bd29e43f1b15e9e068ecb129ed42a97bc248ecf6f5b875c8b7cdd6"} err="failed to get container status \"73d05389b5bd29e43f1b15e9e068ecb129ed42a97bc248ecf6f5b875c8b7cdd6\": rpc error: code = NotFound desc = could not find container \"73d05389b5bd29e43f1b15e9e068ecb129ed42a97bc248ecf6f5b875c8b7cdd6\": container with ID starting with 73d05389b5bd29e43f1b15e9e068ecb129ed42a97bc248ecf6f5b875c8b7cdd6 not found: ID does not exist"
Nov 02 13:55:11.391546 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.391508    4942 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c79bx\" (UniqueName: \"kubernetes.io/projected/fd396f31-6170-4a8b-96ee-358fec42f95a-kube-api-access-c79bx\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:55:11.391546 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.391543    4942 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd396f31-6170-4a8b-96ee-358fec42f95a-maas-api-tls\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:55:11.391756 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.391557    4942 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd396f31-6170-4a8b-96ee-358fec42f95a-metrics-certs\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:55:11.391756 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.391571    4942 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd396f31-6170-4a8b-96ee-358fec42f95a-openshift-service-ca\") on node \"ip-10-0-135-24.us-west-1.compute.internal\" DevicePath \"\""
Nov 02 13:55:11.453035 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.452948    4942 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"]
Nov 02 13:55:11.457882 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.457856    4942 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8869c0de-bc79c6b6c-2jss4"]
Nov 02 13:55:11.542700 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:55:11.542665    4942 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd396f31-6170-4a8b-96ee-358fec42f95a" path="/var/lib/kubelet/pods/fd396f31-6170-4a8b-96ee-358fec42f95a/volumes"
Nov 02 13:55:30.981350 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:55:30.981314    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:56:00.981896 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:56:00.981857    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:56:14.505104 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:56:14.504987    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:56:14.533920 ip-10-0-135-24 kubenswrapper[4942]: I1102 13:56:14.533891    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 13:56:30.982672 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:56:30.982622    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:57:00.982396 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:57:00.982323    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:57:30.985647 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:57:30.985605    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:58:01.050250 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:58:01.050207    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:58:30.983515 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:58:30.983432    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:59:00.983678 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:59:00.983637    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 13:59:30.981934 ip-10-0-135-24 kubenswrapper[4942]: E1102 13:59:30.981898    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 14:00:01.049180 ip-10-0-135-24 kubenswrapper[4942]: E1102 14:00:01.049089    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 14:00:30.982549 ip-10-0-135-24 kubenswrapper[4942]: E1102 14:00:30.982515    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 14:00:34.189451 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:00:34.189413    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-t2q27 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.143:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 14:00:34.190256 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:00:34.190221    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-t2q27" podUID="61489084-5c87-4d75-8ccd-ddeea606747a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.143:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 14:00:34.318322 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:00:34.317956    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-z6h47 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 14:00:34.318322 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:00:34.318025    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-z6h47" podUID="b55accad-1dd4-4a22-8f14-cddd8a0283af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.144:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 14:00:34.844443 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:00:34.844401    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-547v7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.145:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 14:00:34.844635 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:00:34.844478    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7" podUID="4428344b-e3a5-4dd1-b784-6a8d1dec4a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.145:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 14:01:00.983675 ip-10-0-135-24 kubenswrapper[4942]: E1102 14:01:00.983639    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 14:01:04.836500 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:01:04.836460    4942 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6dcf757899-547v7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.145:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 02 14:01:04.837022 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:01:04.836604    4942 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6dcf757899-547v7" podUID="4428344b-e3a5-4dd1-b784-6a8d1dec4a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.145:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 02 14:01:14.637771 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:01:14.637740    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 14:01:14.648778 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:01:14.648756    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2qq64_53ea0d03-1f13-4ee1-af04-9ddf449d6b33/console-operator/2.log"
Nov 02 14:01:30.984813 ip-10-0-135-24 kubenswrapper[4942]: E1102 14:01:30.984719    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 14:01:50.098065 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:01:50.098032    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-fb96bcc76-d9wdp_a8304b1f-5415-4992-a8d5-23e2c27e6b28/authorino/0.log"
Nov 02 14:02:00.982273 ip-10-0-135-24 kubenswrapper[4942]: E1102 14:02:00.982237    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 14:02:03.142897 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:03.142865    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-t5hm9_3e1f269e-665c-4850-a92e-3fc1604c6efb/manager/0.log"
Nov 02 14:02:05.663283 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:05.663239    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6459565456-l82jx_32d13fe9-ef8b-4050-8b97-f52f6035c1d9/maas-api/0.log"
Nov 02 14:02:09.817858 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:09.817824    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-md7fh_783954f3-441b-40cb-aa26-219ed6d52aa7/postgres/0.log"
Nov 02 14:02:11.801234 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:11.801200    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-fb96bcc76-d9wdp_a8304b1f-5415-4992-a8d5-23e2c27e6b28/authorino/0.log"
Nov 02 14:02:12.451122 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:12.451091    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-56ffx_58467a64-e86d-42d0-9ef7-8fcafca6a9e5/manager/0.log"
Nov 02 14:02:12.935345 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:12.935307    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-bsm7c_4f77818a-ad1b-48ec-b8aa-76fce65162f6/manager/0.log"
Nov 02 14:02:13.535657 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:13.535624    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-vktcs_da858d4c-2fba-41ef-a18a-2ee3d0dcbbea/manager/0.log"
Nov 02 14:02:14.577266 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:14.577228    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z_adac8dc5-74ea-423a-9ff3-ab12fe7c6585/istio-proxy/0.log"
Nov 02 14:02:15.481346 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:15.481289    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-547v7_4428344b-e3a5-4dd1-b784-6a8d1dec4a37/istio-proxy/0.log"
Nov 02 14:02:17.286432 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:17.286397    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-t2q27_61489084-5c87-4d75-8ccd-ddeea606747a/istio-proxy/0.log"
Nov 02 14:02:17.512929 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:17.512896    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-z6h47_b55accad-1dd4-4a22-8f14-cddd8a0283af/istio-proxy/0.log"
Nov 02 14:02:17.738829 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:17.738795    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk_284e2d6f-6e51-4053-bf44-1809511df1f5/istio-proxy/0.log"
Nov 02 14:02:18.396570 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:18.396537    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-567988bb57-rzsqv_15291745-62ec-45f2-85f9-373504d79de2/router/0.log"
Nov 02 14:02:21.155925 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:21.155887    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-2hcp9_ded7be50-33c3-48ec-9c1e-cab1aaf939b1/main/0.log"
Nov 02 14:02:21.439762 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:21.439677    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8k5m6q_ebd62eee-d630-4bab-8d45-b766272f482a/main/0.log"
Nov 02 14:02:21.881750 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:21.881716    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-mghxh_430e2338-5daf-4020-9fc0-5645de2bf124/main/0.log"
Nov 02 14:02:22.109076 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:22.109042    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-d54rw_242aa9d0-49d5-4981-ad74-0f7b95635e6a/main/0.log"
Nov 02 14:02:30.982426 ip-10-0-135-24 kubenswrapper[4942]: E1102 14:02:30.982391    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 14:02:33.513337 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:33.513289    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z927c/must-gather-2cj7w"]
Nov 02 14:02:33.513721 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:33.513702    4942 cpu_manager.go:414] "containerMap: removing container" podUID="fd396f31-6170-4a8b-96ee-358fec42f95a" containerName="maas-api"
Nov 02 14:02:33.513721 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:33.513713    4942 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd396f31-6170-4a8b-96ee-358fec42f95a" containerName="maas-api"
Nov 02 14:02:33.513810 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:33.513787    4942 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd396f31-6170-4a8b-96ee-358fec42f95a" containerName="maas-api"
Nov 02 14:02:33.517113 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:33.517094    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z927c/must-gather-2cj7w"
Nov 02 14:02:33.574738 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:33.574711    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z927c/must-gather-2cj7w"]
Nov 02 14:02:33.619232 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:33.619196    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aceec9d0-683f-4c64-a075-4c262c442df0-must-gather-output\") pod \"must-gather-2cj7w\" (UID: \"aceec9d0-683f-4c64-a075-4c262c442df0\") " pod="openshift-must-gather-z927c/must-gather-2cj7w"
Nov 02 14:02:33.619396 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:33.619241    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k67ws\" (UniqueName: \"kubernetes.io/projected/aceec9d0-683f-4c64-a075-4c262c442df0-kube-api-access-k67ws\") pod \"must-gather-2cj7w\" (UID: \"aceec9d0-683f-4c64-a075-4c262c442df0\") " pod="openshift-must-gather-z927c/must-gather-2cj7w"
Nov 02 14:02:33.620917 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:33.620896    4942 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-z927c\"/\"default-dockercfg-b99r6\""
Nov 02 14:02:33.620992 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:33.620896    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z927c\"/\"kube-root-ca.crt\""
Nov 02 14:02:33.620992 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:33.620937    4942 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z927c\"/\"openshift-service-ca.crt\""
Nov 02 14:02:33.720677 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:33.720647    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aceec9d0-683f-4c64-a075-4c262c442df0-must-gather-output\") pod \"must-gather-2cj7w\" (UID: \"aceec9d0-683f-4c64-a075-4c262c442df0\") " pod="openshift-must-gather-z927c/must-gather-2cj7w"
Nov 02 14:02:33.720824 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:33.720692    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k67ws\" (UniqueName: \"kubernetes.io/projected/aceec9d0-683f-4c64-a075-4c262c442df0-kube-api-access-k67ws\") pod \"must-gather-2cj7w\" (UID: \"aceec9d0-683f-4c64-a075-4c262c442df0\") " pod="openshift-must-gather-z927c/must-gather-2cj7w"
Nov 02 14:02:33.720993 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:33.720973    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aceec9d0-683f-4c64-a075-4c262c442df0-must-gather-output\") pod \"must-gather-2cj7w\" (UID: \"aceec9d0-683f-4c64-a075-4c262c442df0\") " pod="openshift-must-gather-z927c/must-gather-2cj7w"
Nov 02 14:02:33.781191 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:33.781126    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k67ws\" (UniqueName: \"kubernetes.io/projected/aceec9d0-683f-4c64-a075-4c262c442df0-kube-api-access-k67ws\") pod \"must-gather-2cj7w\" (UID: \"aceec9d0-683f-4c64-a075-4c262c442df0\") " pod="openshift-must-gather-z927c/must-gather-2cj7w"
Nov 02 14:02:33.830760 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:33.830729    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z927c/must-gather-2cj7w"
Nov 02 14:02:34.207507 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:34.207481    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z927c/must-gather-2cj7w"]
Nov 02 14:02:34.209203 ip-10-0-135-24 kubenswrapper[4942]: W1102 14:02:34.209166    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaceec9d0_683f_4c64_a075_4c262c442df0.slice/crio-4c0ccd7805ee9695cb72bfcd37589490273e2585ac9f80f5bffa21fd881155dd WatchSource:0}: Error finding container 4c0ccd7805ee9695cb72bfcd37589490273e2585ac9f80f5bffa21fd881155dd: Status 404 returned error can't find the container with id 4c0ccd7805ee9695cb72bfcd37589490273e2585ac9f80f5bffa21fd881155dd
Nov 02 14:02:34.211180 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:34.211158    4942 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 02 14:02:34.327185 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:34.327152    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z927c/must-gather-2cj7w" event={"ID":"aceec9d0-683f-4c64-a075-4c262c442df0","Type":"ContainerStarted","Data":"4c0ccd7805ee9695cb72bfcd37589490273e2585ac9f80f5bffa21fd881155dd"}
Nov 02 14:02:36.341366 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:36.341328    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z927c/must-gather-2cj7w" event={"ID":"aceec9d0-683f-4c64-a075-4c262c442df0","Type":"ContainerStarted","Data":"6c64f68fec709b7a978f614b537458792e8918c1b34700451664cb910ea8a1ce"}
Nov 02 14:02:36.341855 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:36.341377    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z927c/must-gather-2cj7w" event={"ID":"aceec9d0-683f-4c64-a075-4c262c442df0","Type":"ContainerStarted","Data":"4818d0b8bc558961ea7751b5bbba41e8c11edb07d865beeded3c8176dadb54de"}
Nov 02 14:02:36.459098 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:36.459036    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-z927c/must-gather-2cj7w" podStartSLOduration=2.182808598 podStartE2EDuration="3.459017926s" totalImagesPullingTime="1.276209328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-02 14:02:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:02:36.458054434 +0000 UTC m=+7583.029404192" watchObservedRunningTime="2026-11-02 14:02:36.459017926 +0000 UTC m=+7583.030367685"
Nov 02 14:02:43.445324 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:43.441289    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vm62x_6d8ba8e4-34a6-47eb-a33f-b67c52acd079/global-pull-secret-syncer/0.log"
Nov 02 14:02:44.032320 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:44.032094    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-7tt7c_cc29f105-14d5-4004-afe0-1d0c8c17a5fb/connection-checker/0.log"
Nov 02 14:02:44.462317 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:44.461932    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-lskz4_269be49a-6e81-4730-960e-cb0b4a7b031b/connection-checker/0.log"
Nov 02 14:02:44.902317 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:44.901617    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-sgrg9_5e041a46-74d9-4fc1-82e7-6c83d2b3c229/connection-checker/0.log"
Nov 02 14:02:45.867322 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:45.864967    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-797nn_6b350fc9-6bb7-43c4-9b89-c5157f42ca26/konnectivity-agent/0.log"
Nov 02 14:02:46.023314 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:46.023270    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-t5hm9_3e1f269e-665c-4850-a92e-3fc1604c6efb/manager/0.log"
Nov 02 14:02:47.130739 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:47.130705    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-24.us-west-1.compute.internal_a8a4e5ba356fe85cd903457b215689b7/haproxy/0.log"
Nov 02 14:02:51.175342 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:51.172495    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-t5hm9_3e1f269e-665c-4850-a92e-3fc1604c6efb/manager/0.log"
Nov 02 14:02:51.566084 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:51.565990    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-fb96bcc76-d9wdp_a8304b1f-5415-4992-a8d5-23e2c27e6b28/authorino/0.log"
Nov 02 14:02:52.855242 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:52.855204    4942 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"]
Nov 02 14:02:52.864131 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:52.864103    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:02:52.918882 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:52.918843    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4fd84cbf-f33a-4bec-ae95-242b40931771-proc\") pod \"perf-node-gather-daemonset-58dch\" (UID: \"4fd84cbf-f33a-4bec-ae95-242b40931771\") " pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:02:52.919045 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:52.918893    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpx2x\" (UniqueName: \"kubernetes.io/projected/4fd84cbf-f33a-4bec-ae95-242b40931771-kube-api-access-vpx2x\") pod \"perf-node-gather-daemonset-58dch\" (UID: \"4fd84cbf-f33a-4bec-ae95-242b40931771\") " pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:02:52.919045 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:52.918975    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fd84cbf-f33a-4bec-ae95-242b40931771-sys\") pod \"perf-node-gather-daemonset-58dch\" (UID: \"4fd84cbf-f33a-4bec-ae95-242b40931771\") " pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:02:52.919134 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:52.919070    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fd84cbf-f33a-4bec-ae95-242b40931771-lib-modules\") pod \"perf-node-gather-daemonset-58dch\" (UID: \"4fd84cbf-f33a-4bec-ae95-242b40931771\") " pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:02:52.919134 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:52.919122    4942 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4fd84cbf-f33a-4bec-ae95-242b40931771-podres\") pod \"perf-node-gather-daemonset-58dch\" (UID: \"4fd84cbf-f33a-4bec-ae95-242b40931771\") " pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:02:52.919881 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:52.919849    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"]
Nov 02 14:02:53.020499 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:53.020462    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4fd84cbf-f33a-4bec-ae95-242b40931771-proc\") pod \"perf-node-gather-daemonset-58dch\" (UID: \"4fd84cbf-f33a-4bec-ae95-242b40931771\") " pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:02:53.020499 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:53.020499    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpx2x\" (UniqueName: \"kubernetes.io/projected/4fd84cbf-f33a-4bec-ae95-242b40931771-kube-api-access-vpx2x\") pod \"perf-node-gather-daemonset-58dch\" (UID: \"4fd84cbf-f33a-4bec-ae95-242b40931771\") " pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:02:53.020772 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:53.020550    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fd84cbf-f33a-4bec-ae95-242b40931771-sys\") pod \"perf-node-gather-daemonset-58dch\" (UID: \"4fd84cbf-f33a-4bec-ae95-242b40931771\") " pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:02:53.020772 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:53.020570    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fd84cbf-f33a-4bec-ae95-242b40931771-lib-modules\") pod \"perf-node-gather-daemonset-58dch\" (UID: \"4fd84cbf-f33a-4bec-ae95-242b40931771\") " pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:02:53.020772 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:53.020588    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4fd84cbf-f33a-4bec-ae95-242b40931771-proc\") pod \"perf-node-gather-daemonset-58dch\" (UID: \"4fd84cbf-f33a-4bec-ae95-242b40931771\") " pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:02:53.020772 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:53.020652    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fd84cbf-f33a-4bec-ae95-242b40931771-lib-modules\") pod \"perf-node-gather-daemonset-58dch\" (UID: \"4fd84cbf-f33a-4bec-ae95-242b40931771\") " pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:02:53.020772 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:53.020716    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fd84cbf-f33a-4bec-ae95-242b40931771-sys\") pod \"perf-node-gather-daemonset-58dch\" (UID: \"4fd84cbf-f33a-4bec-ae95-242b40931771\") " pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:02:53.020772 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:53.020764    4942 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4fd84cbf-f33a-4bec-ae95-242b40931771-podres\") pod \"perf-node-gather-daemonset-58dch\" (UID: \"4fd84cbf-f33a-4bec-ae95-242b40931771\") " pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:02:53.021129 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:53.020832    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4fd84cbf-f33a-4bec-ae95-242b40931771-podres\") pod \"perf-node-gather-daemonset-58dch\" (UID: \"4fd84cbf-f33a-4bec-ae95-242b40931771\") " pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:02:53.085615 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:53.082354    4942 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpx2x\" (UniqueName: \"kubernetes.io/projected/4fd84cbf-f33a-4bec-ae95-242b40931771-kube-api-access-vpx2x\") pod \"perf-node-gather-daemonset-58dch\" (UID: \"4fd84cbf-f33a-4bec-ae95-242b40931771\") " pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:02:53.178633 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:53.178558    4942 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:02:53.565692 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:53.565670    4942 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"]
Nov 02 14:02:53.567744 ip-10-0-135-24 kubenswrapper[4942]: W1102 14:02:53.567716    4942 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4fd84cbf_f33a_4bec_ae95_242b40931771.slice/crio-dc2edd22c0fe5e95fe04ea2a1154d25dfe995d218b4d289493ff4823d81faf54 WatchSource:0}: Error finding container dc2edd22c0fe5e95fe04ea2a1154d25dfe995d218b4d289493ff4823d81faf54: Status 404 returned error can't find the container with id dc2edd22c0fe5e95fe04ea2a1154d25dfe995d218b4d289493ff4823d81faf54
Nov 02 14:02:53.825085 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:53.825008    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-56ffx_58467a64-e86d-42d0-9ef7-8fcafca6a9e5/manager/0.log"
Nov 02 14:02:54.519051 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:54.519016    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch" event={"ID":"4fd84cbf-f33a-4bec-ae95-242b40931771","Type":"ContainerStarted","Data":"c3acb306d7da4b9ca8934cea3d31a00f9bbbdbdab93ecc252e7cf7786c950dad"}
Nov 02 14:02:54.519482 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:54.519072    4942 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch" event={"ID":"4fd84cbf-f33a-4bec-ae95-242b40931771","Type":"ContainerStarted","Data":"dc2edd22c0fe5e95fe04ea2a1154d25dfe995d218b4d289493ff4823d81faf54"}
Nov 02 14:02:54.519482 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:54.519215    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:02:54.635517 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:54.635460    4942 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch" podStartSLOduration=2.635442888 podStartE2EDuration="2.635442888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-02 14:02:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-02 14:02:54.63540918 +0000 UTC m=+7601.206758939" watchObservedRunningTime="2026-11-02 14:02:54.635442888 +0000 UTC m=+7601.206792645"
Nov 02 14:02:54.695400 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:54.695364    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-bsm7c_4f77818a-ad1b-48ec-b8aa-76fce65162f6/manager/0.log"
Nov 02 14:02:55.650971 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:55.650919    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-vktcs_da858d4c-2fba-41ef-a18a-2ee3d0dcbbea/manager/0.log"
Nov 02 14:02:56.573708 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:02:56.573671    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-t5hm9_3e1f269e-665c-4850-a92e-3fc1604c6efb/manager/0.log"
Nov 02 14:03:00.536092 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:03:00.536059    4942 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-z927c/perf-node-gather-daemonset-58dch"
Nov 02 14:03:00.983210 ip-10-0-135-24 kubenswrapper[4942]: E1102 14:03:00.983171    4942 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-krn29" containerName="extractor"
Nov 02 14:03:01.183516 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:03:01.183477    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-2dm77_81050700-fe2d-45db-b66e-c83400cb2d35/cluster-monitoring-operator/0.log"
Nov 02 14:03:02.224153 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:03:02.224117    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z_adac8dc5-74ea-423a-9ff3-ab12fe7c6585/istio-proxy/0.log"
Nov 02 14:03:03.662226 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:03:03.662193    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7pvw8_32151a32-010f-470e-bf7f-80122635fa2e/node-exporter/0.log"
Nov 02 14:03:04.076037 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:03:04.075942    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7pvw8_32151a32-010f-470e-bf7f-80122635fa2e/kube-rbac-proxy/0.log"
Nov 02 14:03:04.495161 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:03:04.495131    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7pvw8_32151a32-010f-470e-bf7f-80122635fa2e/init-textfile/0.log"
Nov 02 14:03:05.690233 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:03:05.690202    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-547v7_4428344b-e3a5-4dd1-b784-6a8d1dec4a37/istio-proxy/0.log"
Nov 02 14:03:07.436900 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:03:07.436870    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-t2q27_61489084-5c87-4d75-8ccd-ddeea606747a/istio-proxy/0.log"
Nov 02 14:03:07.664934 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:03:07.664870    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6dcf757899-z6h47_b55accad-1dd4-4a22-8f14-cddd8a0283af/istio-proxy/0.log"
Nov 02 14:03:07.881503 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:03:07.881455    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-58fd47455f-jc6wk_284e2d6f-6e51-4053-bf44-1809511df1f5/istio-proxy/0.log"
Nov 02 14:03:08.532478 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:03:08.532448    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-567988bb57-rzsqv_15291745-62ec-45f2-85f9-373504d79de2/router/0.log"
Nov 02 14:03:12.640056 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:03:12.640022    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-fb96bcc76-d9wdp_a8304b1f-5415-4992-a8d5-23e2c27e6b28/authorino/0.log"
Nov 02 14:03:13.133078 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:03:13.133041    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-56ffx_58467a64-e86d-42d0-9ef7-8fcafca6a9e5/manager/0.log"
Nov 02 14:03:13.596090 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:03:13.596006    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-bsm7c_4f77818a-ad1b-48ec-b8aa-76fce65162f6/manager/0.log"
Nov 02 14:03:14.120528 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:03:14.120490    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-vktcs_da858d4c-2fba-41ef-a18a-2ee3d0dcbbea/manager/0.log"
Nov 02 14:03:16.398220 ip-10-0-135-24 kubenswrapper[4942]: I1102 14:03:16.398180    4942 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7d699c9f7cnf28z_adac8dc5-74ea-423a-9ff3-ab12fe7c6585/istio-proxy/0.log"