Jul 24 15:43:16.139320 ip-10-0-133-124 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 24 15:43:16.139330 ip-10-0-133-124 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 24 15:43:16.139337 ip-10-0-133-124 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 24 15:43:16.139570 ip-10-0-133-124 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 24 15:43:26.251914 ip-10-0-133-124 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 24 15:43:26.251933 ip-10-0-133-124 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot dd24f04f9af2407790abaf1a1fbc1028 --
Jul 24 15:45:19.278281 ip-10-0-133-124 systemd[1]: Starting Kubernetes Kubelet...
Jul 24 15:45:19.681389 ip-10-0-133-124 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 15:45:19.681389 ip-10-0-133-124 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 24 15:45:19.681389 ip-10-0-133-124 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 15:45:19.681389 ip-10-0-133-124 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 24 15:45:19.681389 ip-10-0-133-124 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 24 15:45:19.682411 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.682282    2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 24 15:45:19.688244 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688212    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:45:19.688244 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688239    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:45:19.688244 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688243    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:45:19.688244 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688248    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:45:19.688244 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688251    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:45:19.688244 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688254    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:45:19.688244 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688257    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688261    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688264    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688267    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688270    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688272    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688275    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688278    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688281    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688284    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688286    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688289    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688292    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688295    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688298    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688300    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688303    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688306    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688308    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688311    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:45:19.688523 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688313    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688316    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688318    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688321    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688324    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688327    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688330    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688332    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688335    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688338    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688340    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688343    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688346    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688348    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688351    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688354    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688357    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688359    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688362    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688365    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:45:19.689013 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688367    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688370    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688372    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688375    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688384    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688387    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688390    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688394    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688396    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688399    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688401    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688406    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688410    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688413    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688416    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688418    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688421    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688423    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688426    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:45:19.689503 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688430    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688433    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688437    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688439    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688443    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688446    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688448    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688452    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688455    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688457    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688461    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688466    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688469    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688473    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688476    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688479    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688482    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688484    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688493    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:45:19.690033 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688497    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688499    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688502    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688504    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688507    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688510    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688513    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688516    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688519    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688522    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688524    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688527    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688530    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688532    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.688535    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689351    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689359    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689363    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689366    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:45:19.690598 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689370    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689373    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689376    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689379    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689381    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689385    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689387    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689390    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689393    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689395    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689398    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689400    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689403    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689407    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689410    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689412    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689415    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689417    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689420    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689422    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:45:19.691081 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689425    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689428    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689430    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689433    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689436    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689438    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689442    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689445    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689448    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689450    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689453    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689456    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689459    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689464    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689468    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689470    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689473    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689476    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689478    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689481    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:45:19.691592 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689483    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689486    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689488    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689491    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689495    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689497    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689501    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689504    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689507    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689509    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689512    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689514    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689517    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689520    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689522    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689525    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689527    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689529    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689532    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689535    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:45:19.692099 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689537    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689540    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689542    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689545    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689548    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689551    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689553    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689556    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689559    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689563    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689565    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689568    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689570    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689573    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689575    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689577    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689580    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689582    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689585    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689588    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:45:19.692603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689590    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689592    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689595    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689599    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689602    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689605    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689608    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689611    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689613    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689616    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689619    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689621    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689625    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689628    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.689635    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690080    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690086    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690088    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690091    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:45:19.693109 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690094    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690096    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690099    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690102    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690104    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690107    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690109    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690112    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690115    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690117    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690120    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690124    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690132    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690137    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690141    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690145    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690148    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690151    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690154    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:45:19.693572 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690157    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690160    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690163    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690165    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690169    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690171    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690174    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690176    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690181    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690185    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690188    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690191    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690194    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690196    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690199    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690202    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690204    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690207    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690209    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:45:19.694053 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690212    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690214    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690217    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690219    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690222    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690224    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690226    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690235    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690238    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690241    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690243    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690246    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690248    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690250    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690253    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690256    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690258    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690261    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690264    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690266    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:45:19.694512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690269    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690277    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690280    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690283    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690285    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690288    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690290    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690293    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690295    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690298    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690300    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690303    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690305    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690308    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690311    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690313    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690316    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690318    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690321    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690323    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:45:19.695070 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690329    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690332    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690334    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690337    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690340    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690343    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690345    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690347    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690350    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690354    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690356    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690359    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690361    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690365    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690367    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690370    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.690372    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690482    2572 flags.go:64] FLAG: --address="0.0.0.0"
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690505    2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690512    2572 flags.go:64] FLAG: --anonymous-auth="true"
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690516    2572 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 24 15:45:19.695573 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690521    2572 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690524    2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690529    2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690533    2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690536    2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690539    2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690543    2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690546    2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690559    2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690562    2572 flags.go:64] FLAG: --cgroup-root=""
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690565    2572 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690568    2572 flags.go:64] FLAG: --client-ca-file=""
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690579    2572 flags.go:64] FLAG: --cloud-provider="external"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690582    2572 flags.go:64] FLAG: --cluster-dns="[]"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690589    2572 flags.go:64] FLAG: --cluster-domain=""
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690592    2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690595    2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690598    2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690602    2572 flags.go:64] FLAG: --container-log-max-files="5"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690606    2572 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690611    2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690614    2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690617    2572 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690620    2572 flags.go:64] FLAG: --contention-profiling="false"
Jul 24 15:45:19.696098 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690624    2572 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690627    2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690631    2572 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690634    2572 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690638    2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690641    2572 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690644    2572 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690647    2572 flags.go:64] FLAG: --enable-load-reader="false"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690650    2572 flags.go:64] FLAG: --enable-server="true"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690653    2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690660    2572 flags.go:64] FLAG: --event-burst="100"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690663    2572 flags.go:64] FLAG: --event-qps="50"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690666    2572 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690669    2572 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690671    2572 flags.go:64] FLAG: --eviction-hard=""
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690675    2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690678    2572 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690681    2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690684    2572 flags.go:64] FLAG: --eviction-soft=""
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690687    2572 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690689    2572 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690694    2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690697    2572 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690700    2572 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690703    2572 flags.go:64] FLAG: --fail-swap-on="true"
Jul 24 15:45:19.696692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690706    2572 flags.go:64] FLAG: --feature-gates=""
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690709    2572 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690712    2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690717    2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690720    2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690723    2572 flags.go:64] FLAG: --healthz-port="10248"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690726    2572 flags.go:64] FLAG: --help="false"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690730    2572 flags.go:64] FLAG: --hostname-override="ip-10-0-133-124.ec2.internal"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690733    2572 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690737    2572 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690739    2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690743    2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690746    2572 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690749    2572 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690752    2572 flags.go:64] FLAG: --image-service-endpoint=""
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690755    2572 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690757    2572 flags.go:64] FLAG: --kube-api-burst="100"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690760    2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690763    2572 flags.go:64] FLAG: --kube-api-qps="50"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690766    2572 flags.go:64] FLAG: --kube-reserved=""
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690769    2572 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690772    2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690775    2572 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690778    2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 24 15:45:19.697343 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690781    2572 flags.go:64] FLAG: --lock-file=""
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690783    2572 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690786    2572 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690789    2572 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690795    2572 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690799    2572 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690802    2572 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690804    2572 flags.go:64] FLAG: --logging-format="text"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690807    2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690810    2572 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690813    2572 flags.go:64] FLAG: --manifest-url=""
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690817    2572 flags.go:64] FLAG: --manifest-url-header=""
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690822    2572 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690825    2572 flags.go:64] FLAG: --max-open-files="1000000"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690829    2572 flags.go:64] FLAG: --max-pods="110"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690833    2572 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690836    2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690839    2572 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690842    2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690845    2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690848    2572 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690851    2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690859    2572 flags.go:64] FLAG: --node-status-max-images="50"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690863    2572 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690866    2572 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 24 15:45:19.697953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690869    2572 flags.go:64] FLAG: --pod-cidr=""
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690871    2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690877    2572 flags.go:64] FLAG: --pod-manifest-path=""
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690880    2572 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690882    2572 flags.go:64] FLAG: --pods-per-core="0"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690885    2572 flags.go:64] FLAG: --port="10250"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690907    2572 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690911    2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c26ea786c5543a8f"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690914    2572 flags.go:64] FLAG: --qos-reserved=""
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690917    2572 flags.go:64] FLAG: --read-only-port="10255"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690920    2572 flags.go:64] FLAG: --register-node="true"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690923    2572 flags.go:64] FLAG: --register-with-taints=""
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690927    2572 flags.go:64] FLAG: --registry-burst="10"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690930    2572 flags.go:64] FLAG: --registry-qps="5"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690934    2572 flags.go:64] FLAG: --reserved-cpus=""
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690936    2572 flags.go:64] FLAG: --reserved-memory=""
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690940    2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690943    2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690948    2572 flags.go:64] FLAG: --rotate-certificates="false"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690950    2572 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690953    2572 flags.go:64] FLAG: --runonce="false"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690956    2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690959    2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690963    2572 flags.go:64] FLAG: --seccomp-default="false"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690966    2572 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 24 15:45:19.698619 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690969    2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690972    2572 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690975    2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690978    2572 flags.go:64] FLAG: --storage-driver-password="root"
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690981    2572 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690984    2572 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690987    2572 flags.go:64] FLAG: --storage-driver-user="root"
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690990    2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690993    2572 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690996    2572 flags.go:64] FLAG: --system-cgroups=""
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.690998    2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.691004    2572 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.691006    2572 flags.go:64] FLAG: --tls-cert-file=""
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.691009    2572 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.691013    2572 flags.go:64] FLAG: --tls-min-version=""
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.691016    2572 flags.go:64] FLAG: --tls-private-key-file=""
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.691019    2572 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.691021    2572 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.691024    2572 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.691027    2572 flags.go:64] FLAG: --v="2"
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.691032    2572 flags.go:64] FLAG: --version="false"
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.691040    2572 flags.go:64] FLAG: --vmodule=""
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.691044    2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.691048    2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691202    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:45:19.699261 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691208    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691211    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691214    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691218    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691220    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691225    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691227    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691230    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691234    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691238    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691240    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691244    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691246    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691249    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691252    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691255    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691258    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691260    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691263    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691265    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:45:19.699876 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691268    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691270    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691273    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691275    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691278    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691280    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691282    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691285    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691287    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691291    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691293    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691297    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691302    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691305    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691308    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691310    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691313    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691316    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691319    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691321    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:45:19.700390 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691324    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691328    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691332    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691335    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691337    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691340    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691343    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691346    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691349    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691352    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691354    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691357    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691360    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691362    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691365    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691367    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691370    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691373    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691375    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:45:19.700884 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691378    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691381    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691384    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691387    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691390    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691394    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691396    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691399    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691401    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691403    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691407    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691410    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691412    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691415    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691418    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691421    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691423    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691426    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691429    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691431    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:45:19.701441 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691434    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:45:19.701941 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691436    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:45:19.701941 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691439    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:45:19.701941 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691441    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:45:19.701941 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691444    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:45:19.701941 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691446    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:45:19.701941 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691449    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:45:19.701941 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691452    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:45:19.701941 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691454    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:45:19.701941 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691457    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:45:19.701941 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691459    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:45:19.701941 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691462    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:45:19.701941 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691465    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:45:19.701941 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691467    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:45:19.701941 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691470    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:45:19.701941 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691473    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:45:19.701941 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691476    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:45:19.701941 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691480    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:45:19.701941 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.691483    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:45:19.702378 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.692150    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 15:45:19.702378 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.699596    2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 24 15:45:19.702378 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.699622    2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 24 15:45:19.702378 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.699636    2572 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 24 15:45:19.702378 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.699642    2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 24 15:45:19.702378 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699713    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:45:19.702378 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699719    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:45:19.702378 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699723    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:45:19.702378 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699726    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:45:19.702378 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699728    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:45:19.702378 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699731    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:45:19.702378 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699734    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:45:19.702378 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699737    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:45:19.702378 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699739    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:45:19.702378 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699742    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699745    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699748    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699751    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699754    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699756    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699759    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699762    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699764    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699767    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699769    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699772    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699774    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699777    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699780    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699786    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699789    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699792    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699794    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699798    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:45:19.702770 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699802    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699807    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699811    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699814    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699817    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699820    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699823    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699825    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699828    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699831    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699833    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699836    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699839    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699841    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699844    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699846    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699849    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699852    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699854    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:45:19.703285 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699857    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699859    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699862    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699865    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699868    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699870    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699873    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699875    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699878    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699882    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699885    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699906    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699910    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699913    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699916    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699919    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699922    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699924    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699927    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699929    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:45:19.703743 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699933    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699935    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699938    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699940    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699943    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699945    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699948    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699950    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699953    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699955    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699958    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699961    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699964    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699967    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699969    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699974    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699977    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699980    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699982    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699985    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:45:19.704328 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699988    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:45:19.704826 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699991    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:45:19.704826 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699995    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:45:19.704826 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.699998    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:45:19.704826 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700000    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:45:19.704826 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700005    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:45:19.704826 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700008    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:45:19.704826 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700012    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:45:19.704826 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700014    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:45:19.704826 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700017    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:45:19.704826 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700020    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:45:19.704826 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.700025    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 15:45:19.704826 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700142    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 24 15:45:19.704826 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700146    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 24 15:45:19.704826 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700171    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 24 15:45:19.704826 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700175    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700178    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700182    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700184    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700188    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700191    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700193    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700196    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700198    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700201    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700203    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700206    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700209    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700212    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700214    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700217    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700220    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700222    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700225    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700227    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 24 15:45:19.705216 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700231    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700235    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700239    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700243    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700246    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700248    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700251    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700254    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700256    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700259    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700261    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700263    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700266    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700268    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700271    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700273    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700275    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700278    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700280    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 24 15:45:19.705713 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700283    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700286    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700288    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700292    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700294    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700297    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700299    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700301    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700304    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700306    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700309    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700311    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700314    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700317    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700320    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700322    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700325    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700327    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700330    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700332    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 24 15:45:19.706295 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700335    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700337    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700339    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700342    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700345    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700347    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700350    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700352    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700355    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700357    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700360    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700363    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700366    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700368    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700371    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700374    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700377    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700379    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700382    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700385    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 24 15:45:19.706962 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700389    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 24 15:45:19.707606 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700392    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 24 15:45:19.707606 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700395    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 24 15:45:19.707606 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700397    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 24 15:45:19.707606 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700400    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 24 15:45:19.707606 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700403    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 24 15:45:19.707606 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700405    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 24 15:45:19.707606 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700409    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 24 15:45:19.707606 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700411    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 24 15:45:19.707606 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700414    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 24 15:45:19.707606 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700418    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 24 15:45:19.707606 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700421    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 24 15:45:19.707606 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700424    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 24 15:45:19.707606 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700427    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 24 15:45:19.707606 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700429    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 24 15:45:19.707606 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700432    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 24 15:45:19.707606 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:19.700435    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 24 15:45:19.708316 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.700439    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 24 15:45:19.708316 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.700579    2572 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 24 15:45:19.708316 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.702511    2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 24 15:45:19.708316 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.703818    2572 server.go:1019] "Starting client certificate rotation"
Jul 24 15:45:19.708316 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.703927    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 15:45:19.708316 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.704435    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 24 15:45:19.708316 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.706123    2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 15:45:19.708862 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.708835    2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 24 15:45:19.717544 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.717513    2572 log.go:25] "Validated CRI v1 runtime API"
Jul 24 15:45:19.721717 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.721696    2572 log.go:25] "Validated CRI v1 image API"
Jul 24 15:45:19.722563 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.722539    2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 24 15:45:19.728393 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.728362    2572 fs.go:135] Filesystem UUIDs: map[68ec782f-9791-43d1-ba6d-ca12f439e445:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 acef71c5-7b20-4bda-a033-cdd1ef7129ac:/dev/nvme0n1p3]
Jul 24 15:45:19.728510 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.728391    2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 24 15:45:19.733742 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.733606    2572 manager.go:216] Machine: {Timestamp:2026-07-24 15:45:19.732158594 +0000 UTC m=+0.333514205 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098504 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27f575142264530f0a7a4e64f191e7 SystemUUID:ec27f575-1422-6453-0f0a-7a4e64f191e7 BootID:dd24f04f-9af2-4077-90ab-af1a1fbc1028 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c0:3b:96:e3:6d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c0:3b:96:e3:6d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:ce:f0:2d:57:8e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 24 15:45:19.733742 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.733730    2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 24 15:45:19.733931 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.733825    2572 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 24 15:45:19.735073 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.735053    2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 15:45:19.736712 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.736674    2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 24 15:45:19.736864 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.736716    2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-124.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 24 15:45:19.737363 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.737350    2572 topology_manager.go:138] "Creating topology manager with none policy"
Jul 24 15:45:19.737403 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.737365    2572 container_manager_linux.go:309] "Creating device plugin manager"
Jul 24 15:45:19.737403 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.737373    2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 15:45:19.737960 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.737947    2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 24 15:45:19.738983 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.738970    2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 24 15:45:19.741563 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.741546    2572 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 15:45:19.741719 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.741706    2572 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 24 15:45:19.741762 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.741753    2572 kubelet.go:486] "Attempting to sync node with API server"
Jul 24 15:45:19.741801 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.741782    2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 24 15:45:19.741832 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.741808    2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 24 15:45:19.741832 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.741823    2572 kubelet.go:398] "Adding apiserver pod source"
Jul 24 15:45:19.741885 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.741836    2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 24 15:45:19.742385 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.742369    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 15:45:19.742385 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.742390    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 24 15:45:19.744562 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.744542    2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 24 15:45:19.745886 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.745868    2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 24 15:45:19.746004 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.745924    2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 24 15:45:19.746461 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.746448    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 24 15:45:19.746522 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.746468    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 24 15:45:19.746522 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.746477    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 24 15:45:19.746522 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.746485    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 24 15:45:19.746522 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.746497    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 24 15:45:19.746522 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.746505    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 24 15:45:19.746759 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.746687    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 24 15:45:19.746759 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.746700    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 24 15:45:19.746759 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.746712    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 24 15:45:19.746759 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.746721    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 24 15:45:19.746759 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.746735    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 24 15:45:19.746759 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.746747    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 24 15:45:19.747537 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.747526    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 24 15:45:19.747594 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.747540    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 24 15:45:19.751001 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.750984    2572 server.go:1268] "Started kubelet"
Jul 24 15:45:19.751101 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.751055    2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 24 15:45:19.752852 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.751081    2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 24 15:45:19.752960 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.752917    2572 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 24 15:45:19.753435 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:19.753405    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-124.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 24 15:45:19.753519 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.753496    2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-124.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 24 15:45:19.753998 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:19.753971    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 24 15:45:19.754439 ip-10-0-133-124 systemd[1]: Started Kubernetes Kubelet.
Jul 24 15:45:19.754706 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.754686    2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 24 15:45:19.757381 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.757364    2572 server.go:310] "Adding debug handlers to kubelet server"
Jul 24 15:45:19.759465 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:19.758573    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-124.ec2.internal.18c5444bd1896c2c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-124.ec2.internal,UID:ip-10-0-133-124.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-124.ec2.internal,},FirstTimestamp:2026-07-24 15:45:19.750949932 +0000 UTC m=+0.352305544,LastTimestamp:2026-07-24 15:45:19.750949932 +0000 UTC m=+0.352305544,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-124.ec2.internal,}"
Jul 24 15:45:19.761008 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:19.760983    2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 24 15:45:19.762554 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.762528    2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 24 15:45:19.762599 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.762556    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 24 15:45:19.762644 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:19.762625    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-124.ec2.internal\" not found"
Jul 24 15:45:19.763095 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.763054    2572 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 24 15:45:19.763095 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.763072    2572 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 24 15:45:19.763095 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.763050    2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 24 15:45:19.763281 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.763179    2572 reconstruct.go:97] "Volume reconstruction finished"
Jul 24 15:45:19.763281 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.763187    2572 reconciler.go:29] "Reconciler: start to sync state"
Jul 24 15:45:19.763397 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.763383    2572 factory.go:55] Registering systemd factory
Jul 24 15:45:19.763456 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.763402    2572 factory.go:223] Registration of the systemd container factory successfully
Jul 24 15:45:19.763787 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.763772    2572 factory.go:153] Registering CRI-O factory
Jul 24 15:45:19.763860 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.763793    2572 factory.go:223] Registration of the crio container factory successfully
Jul 24 15:45:19.763860 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.763849    2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 24 15:45:19.763985 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.763876    2572 factory.go:103] Registering Raw factory
Jul 24 15:45:19.763985 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.763916    2572 manager.go:1196] Started watching for new ooms in manager
Jul 24 15:45:19.764346 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.764330    2572 manager.go:349] Starting recovery of all containers
Jul 24 15:45:19.772066 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:19.772028    2572 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-124.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 24 15:45:19.772066 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:19.772047    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 24 15:45:19.775056 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.775033    2572 manager.go:354] Recovery completed
Jul 24 15:45:19.779628 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.779610    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:45:19.781749 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.781721    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:45:19.781819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.781767    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:45:19.781819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.781778    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:45:19.782421 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.782403    2572 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 24 15:45:19.782421 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.782417    2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 24 15:45:19.782581 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.782438    2572 state_mem.go:36] "Initialized new in-memory state store"
Jul 24 15:45:19.782581 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.782442    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:45:19.783616 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.783596    2572 policy_none.go:49] "None policy: Start"
Jul 24 15:45:19.783702 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.783621    2572 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 24 15:45:19.783702 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.783638    2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 24 15:45:19.784705 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.784689    2572 policy_none.go:47] "Start"
Jul 24 15:45:19.784812 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.784795    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:45:19.784857 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.784850    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:45:19.784909 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.784864    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:45:19.785415 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:19.785315    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-124.ec2.internal.18c5444bd35f67e2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-124.ec2.internal,UID:ip-10-0-133-124.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-124.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-124.ec2.internal,},FirstTimestamp:2026-07-24 15:45:19.781750754 +0000 UTC m=+0.383106369,LastTimestamp:2026-07-24 15:45:19.781750754 +0000 UTC m=+0.383106369,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-124.ec2.internal,}"
Jul 24 15:45:19.787062 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.786877    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fccrf"
Jul 24 15:45:19.793271 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.793240    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fccrf"
Jul 24 15:45:19.793530 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:19.793460    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-124.ec2.internal.18c5444bd35fbbc0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-124.ec2.internal,UID:ip-10-0-133-124.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-124.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-124.ec2.internal,},FirstTimestamp:2026-07-24 15:45:19.781772224 +0000 UTC m=+0.383127836,LastTimestamp:2026-07-24 15:45:19.781772224 +0000 UTC m=+0.383127836,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-124.ec2.internal,}"
Jul 24 15:45:19.821672 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.821647    2572 manager.go:337] "Starting Device Plugin manager"
Jul 24 15:45:19.821821 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:19.821687    2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 24 15:45:19.821821 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.821698    2572 server.go:87] "Starting device plugin registration server"
Jul 24 15:45:19.822022 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.822011    2572 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 24 15:45:19.822075 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.822024    2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 24 15:45:19.822153 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.822135    2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 24 15:45:19.822239 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.822220    2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 24 15:45:19.822239 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.822233    2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 24 15:45:19.822790 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:19.822773    2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 24 15:45:19.822865 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:19.822820    2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-124.ec2.internal\" not found"
Jul 24 15:45:19.898583 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.898535    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 24 15:45:19.899930 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.899911    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 24 15:45:19.899990 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.899955    2572 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 24 15:45:19.899990 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.899984    2572 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 24 15:45:19.900071 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:19.900022    2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 24 15:45:19.902796 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.902773    2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:45:19.923152 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.923111    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:45:19.924116 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.924095    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:45:19.924230 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.924130    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:45:19.924230 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.924140    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:45:19.924230 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.924164    2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-124.ec2.internal"
Jul 24 15:45:19.933941 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:19.933843    2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-124.ec2.internal"
Jul 24 15:45:19.933941 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:19.933875    2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-124.ec2.internal\": node \"ip-10-0-133-124.ec2.internal\" not found"
Jul 24 15:45:19.964512 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:19.964485    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-124.ec2.internal\" not found"
Jul 24 15:45:20.000183 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.000143    2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-124.ec2.internal"]
Jul 24 15:45:20.000335 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.000239    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:45:20.001236 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.001219    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:45:20.001313 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.001253    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:45:20.001313 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.001267    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:45:20.002558 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.002535    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:45:20.002679 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.002663    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal"
Jul 24 15:45:20.002726 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.002697    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:45:20.003415 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.003387    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:45:20.003505 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.003420    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:45:20.003505 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.003435    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:45:20.003505 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.003453    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:45:20.003505 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.003464    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:45:20.003505 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.003438    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:45:20.004512 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.004490    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-124.ec2.internal"
Jul 24 15:45:20.004556 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.004517    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 24 15:45:20.005263 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.005246    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasSufficientMemory"
Jul 24 15:45:20.005316 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.005275    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasNoDiskPressure"
Jul 24 15:45:20.005316 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.005285    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeHasSufficientPID"
Jul 24 15:45:20.030785 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:20.030762    2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-124.ec2.internal\" not found" node="ip-10-0-133-124.ec2.internal"
Jul 24 15:45:20.035505 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:20.035480    2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-124.ec2.internal\" not found" node="ip-10-0-133-124.ec2.internal"
Jul 24 15:45:20.064339 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.064305    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cf09905fff65fcdc788ab624bd938c38-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal\" (UID: \"cf09905fff65fcdc788ab624bd938c38\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal"
Jul 24 15:45:20.064339 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.064337    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf09905fff65fcdc788ab624bd938c38-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal\" (UID: \"cf09905fff65fcdc788ab624bd938c38\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal"
Jul 24 15:45:20.064489 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.064358    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d00c243b91d89928c6de8f1bbae37e8e-config\") pod \"kube-apiserver-proxy-ip-10-0-133-124.ec2.internal\" (UID: \"d00c243b91d89928c6de8f1bbae37e8e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-124.ec2.internal"
Jul 24 15:45:20.065314 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:20.065295    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-124.ec2.internal\" not found"
Jul 24 15:45:20.165417 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:20.165385    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-124.ec2.internal\" not found"
Jul 24 15:45:20.165577 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.165445    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cf09905fff65fcdc788ab624bd938c38-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal\" (UID: \"cf09905fff65fcdc788ab624bd938c38\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal"
Jul 24 15:45:20.165577 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.165495    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cf09905fff65fcdc788ab624bd938c38-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal\" (UID: \"cf09905fff65fcdc788ab624bd938c38\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal"
Jul 24 15:45:20.165577 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.165518    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf09905fff65fcdc788ab624bd938c38-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal\" (UID: \"cf09905fff65fcdc788ab624bd938c38\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal"
Jul 24 15:45:20.165577 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.165536    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d00c243b91d89928c6de8f1bbae37e8e-config\") pod \"kube-apiserver-proxy-ip-10-0-133-124.ec2.internal\" (UID: \"d00c243b91d89928c6de8f1bbae37e8e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-124.ec2.internal"
Jul 24 15:45:20.165577 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.165560    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d00c243b91d89928c6de8f1bbae37e8e-config\") pod \"kube-apiserver-proxy-ip-10-0-133-124.ec2.internal\" (UID: \"d00c243b91d89928c6de8f1bbae37e8e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-124.ec2.internal"
Jul 24 15:45:20.165577 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.165577    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf09905fff65fcdc788ab624bd938c38-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal\" (UID: \"cf09905fff65fcdc788ab624bd938c38\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal"
Jul 24 15:45:20.266077 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:20.266006    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-124.ec2.internal\" not found"
Jul 24 15:45:20.333222 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.333192    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal"
Jul 24 15:45:20.338129 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:20.338111    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-124.ec2.internal"
Jul 24 15:45:20.366837 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:20.366800    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-124.ec2.internal\" not found"
Jul 24 15:45:20.467335 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:20.467278    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-124.ec2.internal\" not found"
Jul 24 15:45:20.567769 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:20.567673    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-124.ec2.internal\" not found"
Jul 24 15:45:21.975404 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:21.975368    2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:45:21.985002 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:21.984972    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-124.ec2.internal\" not found"
Jul 24 15:45:22.016853 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.016830    2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:45:22.020155 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.020139    2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 24 15:45:22.020255 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.020238    2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 15:45:22.020316 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.020283    2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 15:45:22.020316 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.020288    2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 24 15:45:22.059484 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.059454    2572 apiserver.go:52] "Watching apiserver"
Jul 24 15:45:22.077821 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.077792    2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 24 15:45:22.078144 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.078124    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-k49df","openshift-dns/node-resolver-5xwjc","openshift-image-registry/node-ca-4lpwk","openshift-multus/multus-additional-cni-plugins-ck8qc","openshift-multus/multus-w6q8q","openshift-network-operator/iptables-alerter-jz8pt","kube-system/konnectivity-agent-w6mr8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm","openshift-cluster-node-tuning-operator/tuned-ng2nc","openshift-multus/network-metrics-daemon-b86md","openshift-network-diagnostics/network-check-target-b669x","openshift-ovn-kubernetes/ovnkube-node-lj8js"]
Jul 24 15:45:22.079513 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.079497    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:22.079584 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.079560    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:22.079668 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.079649    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 24 15:45:22.079725 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.079714    2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal"
Jul 24 15:45:22.080498 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.080484    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5xwjc"
Jul 24 15:45:22.081505 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.081490    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4lpwk"
Jul 24 15:45:22.082483 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.082466    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.083339 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.083322    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 24 15:45:22.083339 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.083334    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 24 15:45:22.083483 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.083346    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bmltr\""
Jul 24 15:45:22.083616 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.083588    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.083718 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.083695    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-22gft\""
Jul 24 15:45:22.083824 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.083730    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 24 15:45:22.083824 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.083730    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 24 15:45:22.083908 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.083733    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 24 15:45:22.084485 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.084469    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jz8pt"
Jul 24 15:45:22.084861 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.084846    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-c2bc7\""
Jul 24 15:45:22.084962 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.084874    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 24 15:45:22.085022 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.084978    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 24 15:45:22.085079 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.085062    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f6f39b6e-359d-4a0a-9b1e-4b1afa097802-hosts-file\") pod \"node-resolver-5xwjc\" (UID: \"f6f39b6e-359d-4a0a-9b1e-4b1afa097802\") " pod="openshift-dns/node-resolver-5xwjc"
Jul 24 15:45:22.085122 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.085095    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6f39b6e-359d-4a0a-9b1e-4b1afa097802-tmp-dir\") pod \"node-resolver-5xwjc\" (UID: \"f6f39b6e-359d-4a0a-9b1e-4b1afa097802\") " pod="openshift-dns/node-resolver-5xwjc"
Jul 24 15:45:22.085165 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.085125    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-kubelet-config\") pod \"global-pull-secret-syncer-k49df\" (UID: \"79ccf6fc-b5a8-4aff-bcb2-c14898719ac4\") " pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:22.085165 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.085148    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret\") pod \"global-pull-secret-syncer-k49df\" (UID: \"79ccf6fc-b5a8-4aff-bcb2-c14898719ac4\") " pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:22.085256 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.085189    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzj9p\" (UniqueName: \"kubernetes.io/projected/f6f39b6e-359d-4a0a-9b1e-4b1afa097802-kube-api-access-vzj9p\") pod \"node-resolver-5xwjc\" (UID: \"f6f39b6e-359d-4a0a-9b1e-4b1afa097802\") " pod="openshift-dns/node-resolver-5xwjc"
Jul 24 15:45:22.085256 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.085233    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-dbus\") pod \"global-pull-secret-syncer-k49df\" (UID: \"79ccf6fc-b5a8-4aff-bcb2-c14898719ac4\") " pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:22.085349 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.085302    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 24 15:45:22.085349 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.085306    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 24 15:45:22.085546 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.085520    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 24 15:45:22.085602 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.085584    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w6mr8"
Jul 24 15:45:22.086067 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.086053    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-64b9x\""
Jul 24 15:45:22.086722 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.086707    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.087671 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.087654    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.088077 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.088055    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 24 15:45:22.088199 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.088154    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 24 15:45:22.088199 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.088172    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 24 15:45:22.088380 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.088222    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-57hwg\""
Jul 24 15:45:22.088453 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.088441    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 24 15:45:22.088489 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.088476    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 24 15:45:22.089019 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.089005    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mmcmz\""
Jul 24 15:45:22.089274 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.089254    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:22.089328 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.089311    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:22.089951 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.089886    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:22.090021 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.089967    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 24 15:45:22.090021 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.089967    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:22.090135 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.090025    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 24 15:45:22.090135 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.090065    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 24 15:45:22.090235 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.090148    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 24 15:45:22.090322 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.090308    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 24 15:45:22.090411 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.090395    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 24 15:45:22.090475 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.090395    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m5xgj\""
Jul 24 15:45:22.090533 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.090498    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zrnw6\""
Jul 24 15:45:22.091520 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.091499    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.092494 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.092479    2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 24 15:45:22.093897 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.093616    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 24 15:45:22.097303 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.094950    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 15:45:22.097303 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.095101    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 24 15:45:22.097303 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.095106    2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-124.ec2.internal"
Jul 24 15:45:22.097303 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.095127    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 24 15:45:22.097303 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.095564    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 24 15:45:22.097303 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.095652    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 24 15:45:22.097303 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.095769    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal"]
Jul 24 15:45:22.097303 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.095836    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pc56q\""
Jul 24 15:45:22.097303 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.095874    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 24 15:45:22.106186 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.106170    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 24 15:45:22.107395 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.107377    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-124.ec2.internal"]
Jul 24 15:45:22.113551 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.113514    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 15:40:21 +0000 UTC" deadline="2028-01-22 08:07:26.996805715 +0000 UTC"
Jul 24 15:45:22.113551 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.113551    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13120h22m4.883257544s"
Jul 24 15:45:22.117333 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.117317    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qlk7g"
Jul 24 15:45:22.125323 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.125302    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qlk7g"
Jul 24 15:45:22.181127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.181107    2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 24 15:45:22.185457 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185434    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/46f61359-1f52-4ef6-9831-b55a3e3096e5-os-release\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.185676 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185461    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-system-cni-dir\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.185676 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185478    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-multus-cni-dir\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.185676 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185492    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-cnibin\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.185676 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185536    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/50c1abb8-1abe-4f3d-8dea-7dd210b396ee-agent-certs\") pod \"konnectivity-agent-w6mr8\" (UID: \"50c1abb8-1abe-4f3d-8dea-7dd210b396ee\") " pod="kube-system/konnectivity-agent-w6mr8"
Jul 24 15:45:22.185676 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185586    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-tuned\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.185676 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185619    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrxzm\" (UniqueName: \"kubernetes.io/projected/ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94-kube-api-access-vrxzm\") pod \"node-ca-4lpwk\" (UID: \"ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94\") " pod="openshift-image-registry/node-ca-4lpwk"
Jul 24 15:45:22.185676 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185652    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/46f61359-1f52-4ef6-9831-b55a3e3096e5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.185676 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185678    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-hostroot\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.186081 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185713    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-lib-modules\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.186081 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185752    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-dbus\") pod \"global-pull-secret-syncer-k49df\" (UID: \"79ccf6fc-b5a8-4aff-bcb2-c14898719ac4\") " pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:22.186081 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185801    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c86da29a-9f5b-4763-82eb-3f9569146b2e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.186081 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185826    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46f61359-1f52-4ef6-9831-b55a3e3096e5-system-cni-dir\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.186081 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185865    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c49km\" (UniqueName: \"kubernetes.io/projected/f00bbdbf-d8c1-482d-890b-4917961038a5-kube-api-access-c49km\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.186081 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185894    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-dbus\") pod \"global-pull-secret-syncer-k49df\" (UID: \"79ccf6fc-b5a8-4aff-bcb2-c14898719ac4\") " pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:22.186081 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185897    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-kubelet-config\") pod \"global-pull-secret-syncer-k49df\" (UID: \"79ccf6fc-b5a8-4aff-bcb2-c14898719ac4\") " pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:22.186081 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185942    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-sysctl-d\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.186081 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185963    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-kubelet-config\") pod \"global-pull-secret-syncer-k49df\" (UID: \"79ccf6fc-b5a8-4aff-bcb2-c14898719ac4\") " pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:22.186081 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.185966    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-etc-kubernetes\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.186081 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186012    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-run-openvswitch\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.186081 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186029    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-run-ovn\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.186081 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186043    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0e30eb70-1243-4f4b-8a15-5df72a224f51-env-overrides\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.186081 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186063    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-sys\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.186081 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186085    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zvvc\" (UniqueName: \"kubernetes.io/projected/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-kube-api-access-5zvvc\") pod \"network-metrics-daemon-b86md\" (UID: \"c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e\") " pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:22.186678 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186102    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0e30eb70-1243-4f4b-8a15-5df72a224f51-ovnkube-config\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.186678 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186116    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/46f61359-1f52-4ef6-9831-b55a3e3096e5-cni-binary-copy\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.186678 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186149    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-multus-socket-dir-parent\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.186678 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186189    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-host-run-multus-certs\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.186678 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186221    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce828380-8975-40e2-b6ae-093b77698dbe-host-slash\") pod \"iptables-alerter-jz8pt\" (UID: \"ce828380-8975-40e2-b6ae-093b77698dbe\") " pod="openshift-network-operator/iptables-alerter-jz8pt"
Jul 24 15:45:22.186678 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186246    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-log-socket\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.186678 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186272    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f6f39b6e-359d-4a0a-9b1e-4b1afa097802-hosts-file\") pod \"node-resolver-5xwjc\" (UID: \"f6f39b6e-359d-4a0a-9b1e-4b1afa097802\") " pod="openshift-dns/node-resolver-5xwjc"
Jul 24 15:45:22.186678 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186299    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6f39b6e-359d-4a0a-9b1e-4b1afa097802-tmp-dir\") pod \"node-resolver-5xwjc\" (UID: \"f6f39b6e-359d-4a0a-9b1e-4b1afa097802\") " pod="openshift-dns/node-resolver-5xwjc"
Jul 24 15:45:22.186678 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186322    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c86da29a-9f5b-4763-82eb-3f9569146b2e-device-dir\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.186678 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186345    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-cni-bin\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.186678 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186350    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f6f39b6e-359d-4a0a-9b1e-4b1afa097802-hosts-file\") pod \"node-resolver-5xwjc\" (UID: \"f6f39b6e-359d-4a0a-9b1e-4b1afa097802\") " pod="openshift-dns/node-resolver-5xwjc"
Jul 24 15:45:22.186678 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186368    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c86da29a-9f5b-4763-82eb-3f9569146b2e-etc-selinux\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.186678 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186392    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/46f61359-1f52-4ef6-9831-b55a3e3096e5-cnibin\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.186678 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186415    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x4tt\" (UniqueName: \"kubernetes.io/projected/46f61359-1f52-4ef6-9831-b55a3e3096e5-kube-api-access-9x4tt\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.186678 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186438    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-run-systemd\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.186678 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186474    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-var-lib-openvswitch\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.187293 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186495    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-node-log\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.187293 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186514    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/46f61359-1f52-4ef6-9831-b55a3e3096e5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.187293 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186536    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-os-release\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.187293 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186562    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f00bbdbf-d8c1-482d-890b-4917961038a5-cni-binary-copy\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.187293 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186612    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-systemd-units\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.187293 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186671    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-cni-netd\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.187293 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186695    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-host-run-k8s-cni-cncf-io\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.187293 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186735    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-host-var-lib-kubelet\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.187293 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186764    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzj9p\" (UniqueName: \"kubernetes.io/projected/f6f39b6e-359d-4a0a-9b1e-4b1afa097802-kube-api-access-vzj9p\") pod \"node-resolver-5xwjc\" (UID: \"f6f39b6e-359d-4a0a-9b1e-4b1afa097802\") " pod="openshift-dns/node-resolver-5xwjc"
Jul 24 15:45:22.187293 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186799    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c86da29a-9f5b-4763-82eb-3f9569146b2e-registration-dir\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.187293 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186835    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-sysctl-conf\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.187293 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186856    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-host\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.187293 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186877    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3caca464-55f2-4c4d-8773-216a1d9212f6-tmp\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.187293 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186905    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hflb\" (UniqueName: \"kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb\") pod \"network-check-target-b669x\" (UID: \"7f4a7724-28ad-4bdb-938b-058b8e0bbcf5\") " pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:22.187293 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186948    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-host-run-netns\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.187293 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186969    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-modprobe-d\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.187293 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.186992    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f00bbdbf-d8c1-482d-890b-4917961038a5-multus-daemon-config\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.187965 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187028    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0e30eb70-1243-4f4b-8a15-5df72a224f51-ovnkube-script-lib\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.187965 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187053    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret\") pod \"global-pull-secret-syncer-k49df\" (UID: \"79ccf6fc-b5a8-4aff-bcb2-c14898719ac4\") " pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:22.187965 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187075    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-run\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.187965 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187096    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2httw\" (UniqueName: \"kubernetes.io/projected/3caca464-55f2-4c4d-8773-216a1d9212f6-kube-api-access-2httw\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.187965 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187117    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94-host\") pod \"node-ca-4lpwk\" (UID: \"ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94\") " pod="openshift-image-registry/node-ca-4lpwk"
Jul 24 15:45:22.187965 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.187154    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:45:22.187965 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187155    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-host-var-lib-cni-bin\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.187965 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.187221    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret podName:79ccf6fc-b5a8-4aff-bcb2-c14898719ac4 nodeName:}" failed. No retries permitted until 2026-07-24 15:45:22.687204938 +0000 UTC m=+1.971532523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret") pod "global-pull-secret-syncer-k49df" (UID: "79ccf6fc-b5a8-4aff-bcb2-c14898719ac4") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:45:22.187965 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187259    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcc6f\" (UniqueName: \"kubernetes.io/projected/ce828380-8975-40e2-b6ae-093b77698dbe-kube-api-access-bcc6f\") pod \"iptables-alerter-jz8pt\" (UID: \"ce828380-8975-40e2-b6ae-093b77698dbe\") " pod="openshift-network-operator/iptables-alerter-jz8pt"
Jul 24 15:45:22.187965 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187291    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-slash\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.187965 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187350    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/46f61359-1f52-4ef6-9831-b55a3e3096e5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.187965 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187383    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-host-var-lib-cni-multus\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.187965 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187410    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-multus-conf-dir\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.187965 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187425    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ce828380-8975-40e2-b6ae-093b77698dbe-iptables-alerter-script\") pod \"iptables-alerter-jz8pt\" (UID: \"ce828380-8975-40e2-b6ae-093b77698dbe\") " pod="openshift-network-operator/iptables-alerter-jz8pt"
Jul 24 15:45:22.187965 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187439    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-etc-openvswitch\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.187965 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187453    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rzv6\" (UniqueName: \"kubernetes.io/projected/0e30eb70-1243-4f4b-8a15-5df72a224f51-kube-api-access-7rzv6\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.188450 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187468    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t774l\" (UniqueName: \"kubernetes.io/projected/c86da29a-9f5b-4763-82eb-3f9569146b2e-kube-api-access-t774l\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.188450 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187483    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-kubernetes\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.188450 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187499    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs\") pod \"network-metrics-daemon-b86md\" (UID: \"c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e\") " pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:22.188450 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187536    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-run-netns\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.188450 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187559    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.188450 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187590    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-sysconfig\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.188450 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187613    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-var-lib-kubelet\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.188450 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187630    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/50c1abb8-1abe-4f3d-8dea-7dd210b396ee-konnectivity-ca\") pod \"konnectivity-agent-w6mr8\" (UID: \"50c1abb8-1abe-4f3d-8dea-7dd210b396ee\") " pod="kube-system/konnectivity-agent-w6mr8"
Jul 24 15:45:22.188450 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187650    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-kubelet\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.188450 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187669    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-run-ovn-kubernetes\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.188450 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187684    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0e30eb70-1243-4f4b-8a15-5df72a224f51-ovn-node-metrics-cert\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.188450 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187716    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c86da29a-9f5b-4763-82eb-3f9569146b2e-socket-dir\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.188450 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187741    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c86da29a-9f5b-4763-82eb-3f9569146b2e-sys-fs\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.188450 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187768    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-systemd\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.188450 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.187807    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94-serviceca\") pod \"node-ca-4lpwk\" (UID: \"ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94\") " pod="openshift-image-registry/node-ca-4lpwk"
Jul 24 15:45:22.188450 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.188157    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6f39b6e-359d-4a0a-9b1e-4b1afa097802-tmp-dir\") pod \"node-resolver-5xwjc\" (UID: \"f6f39b6e-359d-4a0a-9b1e-4b1afa097802\") " pod="openshift-dns/node-resolver-5xwjc"
Jul 24 15:45:22.193692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.193674    2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 24 15:45:22.196725 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.196710    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzj9p\" (UniqueName: \"kubernetes.io/projected/f6f39b6e-359d-4a0a-9b1e-4b1afa097802-kube-api-access-vzj9p\") pod \"node-resolver-5xwjc\" (UID: \"f6f39b6e-359d-4a0a-9b1e-4b1afa097802\") " pod="openshift-dns/node-resolver-5xwjc"
Jul 24 15:45:22.288585 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288546    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-multus-conf-dir\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.288746 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288599    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ce828380-8975-40e2-b6ae-093b77698dbe-iptables-alerter-script\") pod \"iptables-alerter-jz8pt\" (UID: \"ce828380-8975-40e2-b6ae-093b77698dbe\") " pod="openshift-network-operator/iptables-alerter-jz8pt"
Jul 24 15:45:22.288746 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288625    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-etc-openvswitch\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.288746 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288642    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rzv6\" (UniqueName: \"kubernetes.io/projected/0e30eb70-1243-4f4b-8a15-5df72a224f51-kube-api-access-7rzv6\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.288746 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288639    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-multus-conf-dir\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.288746 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288658    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t774l\" (UniqueName: \"kubernetes.io/projected/c86da29a-9f5b-4763-82eb-3f9569146b2e-kube-api-access-t774l\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.288746 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288680    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-kubernetes\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.288746 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288700    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs\") pod \"network-metrics-daemon-b86md\" (UID: \"c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e\") " pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:22.288746 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288722    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-run-netns\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.288746 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288725    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-etc-openvswitch\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.288746 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288747    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.289248 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288808    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-run-netns\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.289248 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288834    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-sysconfig\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.289248 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288857    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-var-lib-kubelet\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.289248 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288880    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/50c1abb8-1abe-4f3d-8dea-7dd210b396ee-konnectivity-ca\") pod \"konnectivity-agent-w6mr8\" (UID: \"50c1abb8-1abe-4f3d-8dea-7dd210b396ee\") " pod="kube-system/konnectivity-agent-w6mr8"
Jul 24 15:45:22.289248 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288875    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.289248 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288902    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-kubelet\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.289248 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288952    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-kubelet\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.289248 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288965    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-run-ovn-kubernetes\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.289248 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288974    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-sysconfig\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.289248 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.288905    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:45:22.289248 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.288994    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0e30eb70-1243-4f4b-8a15-5df72a224f51-ovn-node-metrics-cert\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.289248 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.289060    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs podName:c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e nodeName:}" failed. No retries permitted until 2026-07-24 15:45:22.789040684 +0000 UTC m=+2.073368283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs") pod "network-metrics-daemon-b86md" (UID: "c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:45:22.289248 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289090    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-var-lib-kubelet\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.289248 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289091    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c86da29a-9f5b-4763-82eb-3f9569146b2e-socket-dir\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.289248 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289129    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c86da29a-9f5b-4763-82eb-3f9569146b2e-sys-fs\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.289248 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289151    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-systemd\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.289248 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289174    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94-serviceca\") pod \"node-ca-4lpwk\" (UID: \"ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94\") " pod="openshift-image-registry/node-ca-4lpwk"
Jul 24 15:45:22.290187 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289187    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c86da29a-9f5b-4763-82eb-3f9569146b2e-socket-dir\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.290187 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289198    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/46f61359-1f52-4ef6-9831-b55a3e3096e5-os-release\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.290187 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289223    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-system-cni-dir\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.290187 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289230    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c86da29a-9f5b-4763-82eb-3f9569146b2e-sys-fs\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.290187 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289244    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-multus-cni-dir\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.290187 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289265    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-cnibin\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.290187 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289289    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/50c1abb8-1abe-4f3d-8dea-7dd210b396ee-agent-certs\") pod \"konnectivity-agent-w6mr8\" (UID: \"50c1abb8-1abe-4f3d-8dea-7dd210b396ee\") " pod="kube-system/konnectivity-agent-w6mr8"
Jul 24 15:45:22.290187 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289308    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/46f61359-1f52-4ef6-9831-b55a3e3096e5-os-release\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.290187 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289314    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-tuned\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.290187 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289348    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrxzm\" (UniqueName: \"kubernetes.io/projected/ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94-kube-api-access-vrxzm\") pod \"node-ca-4lpwk\" (UID: \"ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94\") " pod="openshift-image-registry/node-ca-4lpwk"
Jul 24 15:45:22.290187 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289366    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ce828380-8975-40e2-b6ae-093b77698dbe-iptables-alerter-script\") pod \"iptables-alerter-jz8pt\" (UID: \"ce828380-8975-40e2-b6ae-093b77698dbe\") " pod="openshift-network-operator/iptables-alerter-jz8pt"
Jul 24 15:45:22.290187 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289379    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/46f61359-1f52-4ef6-9831-b55a3e3096e5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.290187 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289391    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-run-ovn-kubernetes\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.290187 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289408    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-hostroot\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.290187 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289432    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-lib-modules\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.290187 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289456    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-multus-cni-dir\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.290187 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289483    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c86da29a-9f5b-4763-82eb-3f9569146b2e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.291051 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289488    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-system-cni-dir\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.291051 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289512    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-cnibin\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.291051 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289538    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c86da29a-9f5b-4763-82eb-3f9569146b2e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.291051 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289552    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-hostroot\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.291051 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289581    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46f61359-1f52-4ef6-9831-b55a3e3096e5-system-cni-dir\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.291051 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289589    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-systemd\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.291051 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289604    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c49km\" (UniqueName: \"kubernetes.io/projected/f00bbdbf-d8c1-482d-890b-4917961038a5-kube-api-access-c49km\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.291051 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289636    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-sysctl-d\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.291051 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289637    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/50c1abb8-1abe-4f3d-8dea-7dd210b396ee-konnectivity-ca\") pod \"konnectivity-agent-w6mr8\" (UID: \"50c1abb8-1abe-4f3d-8dea-7dd210b396ee\") " pod="kube-system/konnectivity-agent-w6mr8"
Jul 24 15:45:22.291051 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289689    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-etc-kubernetes\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.291051 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289661    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-etc-kubernetes\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.291051 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289743    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-run-openvswitch\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.291051 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289769    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-run-ovn\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.291051 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289793    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0e30eb70-1243-4f4b-8a15-5df72a224f51-env-overrides\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.291051 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289809    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-lib-modules\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.291051 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289816    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-sys\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.291051 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289842    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zvvc\" (UniqueName: \"kubernetes.io/projected/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-kube-api-access-5zvvc\") pod \"network-metrics-daemon-b86md\" (UID: \"c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e\") " pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:22.291051 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289895    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0e30eb70-1243-4f4b-8a15-5df72a224f51-ovnkube-config\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.291984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289942    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/46f61359-1f52-4ef6-9831-b55a3e3096e5-cni-binary-copy\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.291984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.289970    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-multus-socket-dir-parent\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.291984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290009    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94-serviceca\") pod \"node-ca-4lpwk\" (UID: \"ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94\") " pod="openshift-image-registry/node-ca-4lpwk"
Jul 24 15:45:22.291984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290069    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-sys\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.291984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290102    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-host-run-multus-certs\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.291984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290129    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce828380-8975-40e2-b6ae-093b77698dbe-host-slash\") pod \"iptables-alerter-jz8pt\" (UID: \"ce828380-8975-40e2-b6ae-093b77698dbe\") " pod="openshift-network-operator/iptables-alerter-jz8pt"
Jul 24 15:45:22.291984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290154    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-log-socket\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.291984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290211    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-multus-socket-dir-parent\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.291984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290238    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce828380-8975-40e2-b6ae-093b77698dbe-host-slash\") pod \"iptables-alerter-jz8pt\" (UID: \"ce828380-8975-40e2-b6ae-093b77698dbe\") " pod="openshift-network-operator/iptables-alerter-jz8pt"
Jul 24 15:45:22.291984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290248    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-log-socket\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.291984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290294    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-host-run-multus-certs\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.291984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290323    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c86da29a-9f5b-4763-82eb-3f9569146b2e-device-dir\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.291984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290348    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-cni-bin\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.291984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290380    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c86da29a-9f5b-4763-82eb-3f9569146b2e-etc-selinux\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.291984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290403    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/46f61359-1f52-4ef6-9831-b55a3e3096e5-cnibin\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.291984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290434    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x4tt\" (UniqueName: \"kubernetes.io/projected/46f61359-1f52-4ef6-9831-b55a3e3096e5-kube-api-access-9x4tt\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.291984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290462    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-run-systemd\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.292819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290464    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0e30eb70-1243-4f4b-8a15-5df72a224f51-env-overrides\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.292819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290506    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-var-lib-openvswitch\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.292819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290524    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/46f61359-1f52-4ef6-9831-b55a3e3096e5-cni-binary-copy\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.292819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290530    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-node-log\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.292819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290508    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/46f61359-1f52-4ef6-9831-b55a3e3096e5-cnibin\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.292819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290601    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c86da29a-9f5b-4763-82eb-3f9569146b2e-etc-selinux\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.292819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290603    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-node-log\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.292819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290629    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/46f61359-1f52-4ef6-9831-b55a3e3096e5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.292819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290687    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-os-release\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.292819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290711    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f00bbdbf-d8c1-482d-890b-4917961038a5-cni-binary-copy\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.292819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290725    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-run-openvswitch\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.292819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290804    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-systemd-units\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.292819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290769    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-systemd-units\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.292819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.290858    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46f61359-1f52-4ef6-9831-b55a3e3096e5-system-cni-dir\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.292819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291321    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/46f61359-1f52-4ef6-9831-b55a3e3096e5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.292819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291366    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-cni-netd\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.292819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291397    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-host-run-k8s-cni-cncf-io\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.293490 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291421    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-host-var-lib-kubelet\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.293490 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291428    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/46f61359-1f52-4ef6-9831-b55a3e3096e5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.293490 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291370    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0e30eb70-1243-4f4b-8a15-5df72a224f51-ovnkube-config\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.293490 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291648    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c86da29a-9f5b-4763-82eb-3f9569146b2e-registration-dir\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.293490 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291684    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-sysctl-conf\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.293490 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291691    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-var-lib-openvswitch\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.293490 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291727    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-host\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.293490 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291746    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c86da29a-9f5b-4763-82eb-3f9569146b2e-registration-dir\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.293490 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291751    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3caca464-55f2-4c4d-8773-216a1d9212f6-tmp\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.293490 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291775    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hflb\" (UniqueName: \"kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb\") pod \"network-check-target-b669x\" (UID: \"7f4a7724-28ad-4bdb-938b-058b8e0bbcf5\") " pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:22.293490 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291784    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-cni-netd\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.293490 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291692    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-run-systemd\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.293490 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291746    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-os-release\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.293490 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291834    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c86da29a-9f5b-4763-82eb-3f9569146b2e-device-dir\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.293490 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291891    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-cni-bin\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.293490 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.291941    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-kubernetes\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.293490 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292202    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-sysctl-d\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.294127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292255    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-host-var-lib-kubelet\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.294127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292296    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-run-ovn\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.294127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292331    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-host-run-k8s-cni-cncf-io\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.294127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292425    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-host-run-netns\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.294127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292430    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-sysctl-conf\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.294127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292460    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-modprobe-d\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.294127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292478    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-host-run-netns\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.294127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292487    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f00bbdbf-d8c1-482d-890b-4917961038a5-multus-daemon-config\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.294127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292511    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0e30eb70-1243-4f4b-8a15-5df72a224f51-ovnkube-script-lib\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.294127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292520    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-host\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.294127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292553    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-run\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.294127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292578    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2httw\" (UniqueName: \"kubernetes.io/projected/3caca464-55f2-4c4d-8773-216a1d9212f6-kube-api-access-2httw\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.294127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292592    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-modprobe-d\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.294127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292605    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94-host\") pod \"node-ca-4lpwk\" (UID: \"ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94\") " pod="openshift-image-registry/node-ca-4lpwk"
Jul 24 15:45:22.294127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292630    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-host-var-lib-cni-bin\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.294127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292650    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3caca464-55f2-4c4d-8773-216a1d9212f6-run\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.294127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292654    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcc6f\" (UniqueName: \"kubernetes.io/projected/ce828380-8975-40e2-b6ae-093b77698dbe-kube-api-access-bcc6f\") pod \"iptables-alerter-jz8pt\" (UID: \"ce828380-8975-40e2-b6ae-093b77698dbe\") " pod="openshift-network-operator/iptables-alerter-jz8pt"
Jul 24 15:45:22.294127 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292684    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-slash\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.294671 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292700    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94-host\") pod \"node-ca-4lpwk\" (UID: \"ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94\") " pod="openshift-image-registry/node-ca-4lpwk"
Jul 24 15:45:22.294671 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292711    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/46f61359-1f52-4ef6-9831-b55a3e3096e5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.294671 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292735    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-host-var-lib-cni-multus\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.294671 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292811    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-host-var-lib-cni-multus\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.294671 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292857    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f00bbdbf-d8c1-482d-890b-4917961038a5-host-var-lib-cni-bin\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.294671 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292858    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f00bbdbf-d8c1-482d-890b-4917961038a5-cni-binary-copy\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.294671 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.292952    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0e30eb70-1243-4f4b-8a15-5df72a224f51-host-slash\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.294671 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.293028    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f00bbdbf-d8c1-482d-890b-4917961038a5-multus-daemon-config\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.294671 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.293110    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/46f61359-1f52-4ef6-9831-b55a3e3096e5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.294671 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.293216    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0e30eb70-1243-4f4b-8a15-5df72a224f51-ovnkube-script-lib\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.294671 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.293347    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/50c1abb8-1abe-4f3d-8dea-7dd210b396ee-agent-certs\") pod \"konnectivity-agent-w6mr8\" (UID: \"50c1abb8-1abe-4f3d-8dea-7dd210b396ee\") " pod="kube-system/konnectivity-agent-w6mr8"
Jul 24 15:45:22.294671 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.293628    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3caca464-55f2-4c4d-8773-216a1d9212f6-etc-tuned\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.294671 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.294079    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0e30eb70-1243-4f4b-8a15-5df72a224f51-ovn-node-metrics-cert\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.295166 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.294846    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3caca464-55f2-4c4d-8773-216a1d9212f6-tmp\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.297822 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.297692    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:45:22.298142 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.297756    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:45:22.298348 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.298285    2572 projected.go:196] Error preparing data for projected volume kube-api-access-4hflb for pod openshift-network-diagnostics/network-check-target-b669x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:45:22.298693 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.298560    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb podName:7f4a7724-28ad-4bdb-938b-058b8e0bbcf5 nodeName:}" failed. No retries permitted until 2026-07-24 15:45:22.798447984 +0000 UTC m=+2.082775598 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4hflb" (UniqueName: "kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb") pod "network-check-target-b669x" (UID: "7f4a7724-28ad-4bdb-938b-058b8e0bbcf5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:45:22.299414 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.299365    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrxzm\" (UniqueName: \"kubernetes.io/projected/ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94-kube-api-access-vrxzm\") pod \"node-ca-4lpwk\" (UID: \"ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94\") " pod="openshift-image-registry/node-ca-4lpwk"
Jul 24 15:45:22.299689 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.299651    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t774l\" (UniqueName: \"kubernetes.io/projected/c86da29a-9f5b-4763-82eb-3f9569146b2e-kube-api-access-t774l\") pod \"aws-ebs-csi-driver-node-4tlzm\" (UID: \"c86da29a-9f5b-4763-82eb-3f9569146b2e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.299850 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.299829    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x4tt\" (UniqueName: \"kubernetes.io/projected/46f61359-1f52-4ef6-9831-b55a3e3096e5-kube-api-access-9x4tt\") pod \"multus-additional-cni-plugins-ck8qc\" (UID: \"46f61359-1f52-4ef6-9831-b55a3e3096e5\") " pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.300407 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.300383    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zvvc\" (UniqueName: \"kubernetes.io/projected/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-kube-api-access-5zvvc\") pod \"network-metrics-daemon-b86md\" (UID: \"c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e\") " pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:22.300775 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.300742    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rzv6\" (UniqueName: \"kubernetes.io/projected/0e30eb70-1243-4f4b-8a15-5df72a224f51-kube-api-access-7rzv6\") pod \"ovnkube-node-lj8js\" (UID: \"0e30eb70-1243-4f4b-8a15-5df72a224f51\") " pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.301219 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.301202    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c49km\" (UniqueName: \"kubernetes.io/projected/f00bbdbf-d8c1-482d-890b-4917961038a5-kube-api-access-c49km\") pod \"multus-w6q8q\" (UID: \"f00bbdbf-d8c1-482d-890b-4917961038a5\") " pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.301359 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.301342    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2httw\" (UniqueName: \"kubernetes.io/projected/3caca464-55f2-4c4d-8773-216a1d9212f6-kube-api-access-2httw\") pod \"tuned-ng2nc\" (UID: \"3caca464-55f2-4c4d-8773-216a1d9212f6\") " pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.302080 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.302061    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcc6f\" (UniqueName: \"kubernetes.io/projected/ce828380-8975-40e2-b6ae-093b77698dbe-kube-api-access-bcc6f\") pod \"iptables-alerter-jz8pt\" (UID: \"ce828380-8975-40e2-b6ae-093b77698dbe\") " pod="openshift-network-operator/iptables-alerter-jz8pt"
Jul 24 15:45:22.407814 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.407751    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5xwjc"
Jul 24 15:45:22.426503 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.426472    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4lpwk"
Jul 24 15:45:22.432221 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:22.432190    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf09905fff65fcdc788ab624bd938c38.slice/crio-ab338cbbf6ac368d4b86b6ba8a1eb15b0beb49f02f4a591c08be6170908df017 WatchSource:0}: Error finding container ab338cbbf6ac368d4b86b6ba8a1eb15b0beb49f02f4a591c08be6170908df017: Status 404 returned error can't find the container with id ab338cbbf6ac368d4b86b6ba8a1eb15b0beb49f02f4a591c08be6170908df017
Jul 24 15:45:22.432649 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:22.432627    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6f39b6e_359d_4a0a_9b1e_4b1afa097802.slice/crio-7362f88142c3b326ba650df6f756e0fe5e4be6e2bdc0f1f7493b2617f0a0bc96 WatchSource:0}: Error finding container 7362f88142c3b326ba650df6f756e0fe5e4be6e2bdc0f1f7493b2617f0a0bc96: Status 404 returned error can't find the container with id 7362f88142c3b326ba650df6f756e0fe5e4be6e2bdc0f1f7493b2617f0a0bc96
Jul 24 15:45:22.433105 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:22.433087    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd00c243b91d89928c6de8f1bbae37e8e.slice/crio-9ca73592444c31dc61ee9dd26f1c7f232135f30371bfc24d2a6c379ee15f1841 WatchSource:0}: Error finding container 9ca73592444c31dc61ee9dd26f1c7f232135f30371bfc24d2a6c379ee15f1841: Status 404 returned error can't find the container with id 9ca73592444c31dc61ee9dd26f1c7f232135f30371bfc24d2a6c379ee15f1841
Jul 24 15:45:22.438001 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.437866    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:45:22.438438 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:22.438414    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef6c9d06_124e_4ca0_8bb4_dacf53bb2b94.slice/crio-5342c1e7c658c51caf6bcdc5c58afcc47ebc3effaa452ae281079ca41f4b65d8 WatchSource:0}: Error finding container 5342c1e7c658c51caf6bcdc5c58afcc47ebc3effaa452ae281079ca41f4b65d8: Status 404 returned error can't find the container with id 5342c1e7c658c51caf6bcdc5c58afcc47ebc3effaa452ae281079ca41f4b65d8
Jul 24 15:45:22.440470 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.440419    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ck8qc"
Jul 24 15:45:22.450589 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:22.450569    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46f61359_1f52_4ef6_9831_b55a3e3096e5.slice/crio-b5c4e7b743107daea54efd6c5c99fe339795e8d89387cc38a51bf45e222fc01f WatchSource:0}: Error finding container b5c4e7b743107daea54efd6c5c99fe339795e8d89387cc38a51bf45e222fc01f: Status 404 returned error can't find the container with id b5c4e7b743107daea54efd6c5c99fe339795e8d89387cc38a51bf45e222fc01f
Jul 24 15:45:22.458869 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.458846    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w6q8q"
Jul 24 15:45:22.466368 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:22.466347    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf00bbdbf_d8c1_482d_890b_4917961038a5.slice/crio-c6874ae3366a574101dee5b6facea916ad594f24257363056b3891751073bd87 WatchSource:0}: Error finding container c6874ae3366a574101dee5b6facea916ad594f24257363056b3891751073bd87: Status 404 returned error can't find the container with id c6874ae3366a574101dee5b6facea916ad594f24257363056b3891751073bd87
Jul 24 15:45:22.479873 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.479854    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jz8pt"
Jul 24 15:45:22.487512 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:22.487494    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce828380_8975_40e2_b6ae_093b77698dbe.slice/crio-87128ce6ce755bac2e389fba6fad939b939a7b2a5261bd651d14161187afab6b WatchSource:0}: Error finding container 87128ce6ce755bac2e389fba6fad939b939a7b2a5261bd651d14161187afab6b: Status 404 returned error can't find the container with id 87128ce6ce755bac2e389fba6fad939b939a7b2a5261bd651d14161187afab6b
Jul 24 15:45:22.503719 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.503693    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w6mr8"
Jul 24 15:45:22.512017 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:22.511993    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50c1abb8_1abe_4f3d_8dea_7dd210b396ee.slice/crio-bc1b95e97a0e1db608620f552e3ba7748310d9538cc35e21ac01aee210e411b8 WatchSource:0}: Error finding container bc1b95e97a0e1db608620f552e3ba7748310d9538cc35e21ac01aee210e411b8: Status 404 returned error can't find the container with id bc1b95e97a0e1db608620f552e3ba7748310d9538cc35e21ac01aee210e411b8
Jul 24 15:45:22.526489 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.526463    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm"
Jul 24 15:45:22.536128 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:22.536106    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc86da29a_9f5b_4763_82eb_3f9569146b2e.slice/crio-323189cd27ad3073618d28d125d2823dfb2c1e87ceca036ea4facd2d19cd9e17 WatchSource:0}: Error finding container 323189cd27ad3073618d28d125d2823dfb2c1e87ceca036ea4facd2d19cd9e17: Status 404 returned error can't find the container with id 323189cd27ad3073618d28d125d2823dfb2c1e87ceca036ea4facd2d19cd9e17
Jul 24 15:45:22.537773 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.537755    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ng2nc"
Jul 24 15:45:22.539826 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.539810    2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:45:22.543050 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.543035    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:22.546583 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:22.546525    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3caca464_55f2_4c4d_8773_216a1d9212f6.slice/crio-c10b82c17d87a1bbef69faad056986fbc1f2af2530a214c17aa16db936457fef WatchSource:0}: Error finding container c10b82c17d87a1bbef69faad056986fbc1f2af2530a214c17aa16db936457fef: Status 404 returned error can't find the container with id c10b82c17d87a1bbef69faad056986fbc1f2af2530a214c17aa16db936457fef
Jul 24 15:45:22.551339 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:45:22.551317    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e30eb70_1243_4f4b_8a15_5df72a224f51.slice/crio-1a34219871bce75cd1565db3f9451181d3cc349828f5964f7b9810866a5de9ab WatchSource:0}: Error finding container 1a34219871bce75cd1565db3f9451181d3cc349828f5964f7b9810866a5de9ab: Status 404 returned error can't find the container with id 1a34219871bce75cd1565db3f9451181d3cc349828f5964f7b9810866a5de9ab
Jul 24 15:45:22.552631 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.552610    2572 kuberuntime_manager.go:1459] "Unhandled Error" err="container ovn-controller start failed in pod ovnkube-node-lj8js_openshift-ovn-kubernetes(0e30eb70-1243-4f4b-8a15-5df72a224f51): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 24 15:45:22.554251 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.554235    2572 kuberuntime_manager.go:1459] "Unhandled Error" err="container kube-rbac-proxy-node start failed in pod ovnkube-node-lj8js_openshift-ovn-kubernetes(0e30eb70-1243-4f4b-8a15-5df72a224f51): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 24 15:45:22.557295 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.557256    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js" podUID="0e30eb70-1243-4f4b-8a15-5df72a224f51"
Jul 24 15:45:22.695399 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.695302    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret\") pod \"global-pull-secret-syncer-k49df\" (UID: \"79ccf6fc-b5a8-4aff-bcb2-c14898719ac4\") " pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:22.695544 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.695448    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:45:22.695544 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.695513    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret podName:79ccf6fc-b5a8-4aff-bcb2-c14898719ac4 nodeName:}" failed. No retries permitted until 2026-07-24 15:45:23.695497296 +0000 UTC m=+2.979824885 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret") pod "global-pull-secret-syncer-k49df" (UID: "79ccf6fc-b5a8-4aff-bcb2-c14898719ac4") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:45:22.795936 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.795897    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs\") pod \"network-metrics-daemon-b86md\" (UID: \"c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e\") " pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:22.796092 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.796042    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:45:22.796133 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.796112    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs podName:c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e nodeName:}" failed. No retries permitted until 2026-07-24 15:45:23.796094874 +0000 UTC m=+3.080422460 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs") pod "network-metrics-daemon-b86md" (UID: "c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:45:22.896971 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:22.896933    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hflb\" (UniqueName: \"kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb\") pod \"network-check-target-b669x\" (UID: \"7f4a7724-28ad-4bdb-938b-058b8e0bbcf5\") " pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:22.897151 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.897117    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:45:22.897151 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.897137    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:45:22.897151 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.897149    2572 projected.go:196] Error preparing data for projected volume kube-api-access-4hflb for pod openshift-network-diagnostics/network-check-target-b669x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:45:22.897318 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:22.897205    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb podName:7f4a7724-28ad-4bdb-938b-058b8e0bbcf5 nodeName:}" failed. No retries permitted until 2026-07-24 15:45:23.897187659 +0000 UTC m=+3.181515259 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4hflb" (UniqueName: "kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb") pod "network-check-target-b669x" (UID: "7f4a7724-28ad-4bdb-938b-058b8e0bbcf5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:45:23.127215 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:23.127096    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 15:40:22 +0000 UTC" deadline="2028-03-14 03:53:12.742578598 +0000 UTC"
Jul 24 15:45:23.127215 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:23.127168    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14364h7m49.615415667s"
Jul 24 15:45:23.241998 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:23.241938    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5xwjc" event={"ID":"f6f39b6e-359d-4a0a-9b1e-4b1afa097802","Type":"ContainerStarted","Data":"7362f88142c3b326ba650df6f756e0fe5e4be6e2bdc0f1f7493b2617f0a0bc96"}
Jul 24 15:45:23.270498 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:23.270296    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-124.ec2.internal" event={"ID":"d00c243b91d89928c6de8f1bbae37e8e","Type":"ContainerStarted","Data":"9ca73592444c31dc61ee9dd26f1c7f232135f30371bfc24d2a6c379ee15f1841"}
Jul 24 15:45:23.288211 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:23.288170    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm" event={"ID":"c86da29a-9f5b-4763-82eb-3f9569146b2e","Type":"ContainerStarted","Data":"323189cd27ad3073618d28d125d2823dfb2c1e87ceca036ea4facd2d19cd9e17"}
Jul 24 15:45:23.292746 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:23.292711    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w6q8q" event={"ID":"f00bbdbf-d8c1-482d-890b-4917961038a5","Type":"ContainerStarted","Data":"c6874ae3366a574101dee5b6facea916ad594f24257363056b3891751073bd87"}
Jul 24 15:45:23.308024 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:23.307949    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal" event={"ID":"cf09905fff65fcdc788ab624bd938c38","Type":"ContainerStarted","Data":"ab338cbbf6ac368d4b86b6ba8a1eb15b0beb49f02f4a591c08be6170908df017"}
Jul 24 15:45:23.328511 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:23.327526    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js" event={"ID":"0e30eb70-1243-4f4b-8a15-5df72a224f51","Type":"ContainerStarted","Data":"1a34219871bce75cd1565db3f9451181d3cc349828f5964f7b9810866a5de9ab"}
Jul 24 15:45:23.333061 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:23.332887    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ng2nc" event={"ID":"3caca464-55f2-4c4d-8773-216a1d9212f6","Type":"ContainerStarted","Data":"c10b82c17d87a1bbef69faad056986fbc1f2af2530a214c17aa16db936457fef"}
Jul 24 15:45:23.349186 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:23.348013    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w6mr8" event={"ID":"50c1abb8-1abe-4f3d-8dea-7dd210b396ee","Type":"ContainerStarted","Data":"bc1b95e97a0e1db608620f552e3ba7748310d9538cc35e21ac01aee210e411b8"}
Jul 24 15:45:23.366402 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:23.365269    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:79363f7abd34060a04015eceb060626d31d91d8c0aa51ce25ec751c237169fdc\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:c61b1bd630d745ebd63d45ede0bae739076b4213e12bea2e0bb124e8f4727ab0\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js" podUID="0e30eb70-1243-4f4b-8a15-5df72a224f51"
Jul 24 15:45:23.367628 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:23.367591    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jz8pt" event={"ID":"ce828380-8975-40e2-b6ae-093b77698dbe","Type":"ContainerStarted","Data":"87128ce6ce755bac2e389fba6fad939b939a7b2a5261bd651d14161187afab6b"}
Jul 24 15:45:23.379432 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:23.379358    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ck8qc" event={"ID":"46f61359-1f52-4ef6-9831-b55a3e3096e5","Type":"ContainerStarted","Data":"b5c4e7b743107daea54efd6c5c99fe339795e8d89387cc38a51bf45e222fc01f"}
Jul 24 15:45:23.385075 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:23.385041    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4lpwk" event={"ID":"ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94","Type":"ContainerStarted","Data":"5342c1e7c658c51caf6bcdc5c58afcc47ebc3effaa452ae281079ca41f4b65d8"}
Jul 24 15:45:23.554763 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:23.554731    2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:45:23.704036 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:23.703942    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret\") pod \"global-pull-secret-syncer-k49df\" (UID: \"79ccf6fc-b5a8-4aff-bcb2-c14898719ac4\") " pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:23.704203 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:23.704140    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:45:23.704256 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:23.704209    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret podName:79ccf6fc-b5a8-4aff-bcb2-c14898719ac4 nodeName:}" failed. No retries permitted until 2026-07-24 15:45:25.704190752 +0000 UTC m=+4.988518349 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret") pod "global-pull-secret-syncer-k49df" (UID: "79ccf6fc-b5a8-4aff-bcb2-c14898719ac4") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:45:23.804816 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:23.804484    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs\") pod \"network-metrics-daemon-b86md\" (UID: \"c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e\") " pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:23.804816 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:23.804698    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:45:23.804816 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:23.804805    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs podName:c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e nodeName:}" failed. No retries permitted until 2026-07-24 15:45:25.804757122 +0000 UTC m=+5.089084744 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs") pod "network-metrics-daemon-b86md" (UID: "c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:45:23.905685 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:23.905640    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hflb\" (UniqueName: \"kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb\") pod \"network-check-target-b669x\" (UID: \"7f4a7724-28ad-4bdb-938b-058b8e0bbcf5\") " pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:23.905953 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:23.905891    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:45:23.905953 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:23.905943    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:45:23.906078 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:23.905956    2572 projected.go:196] Error preparing data for projected volume kube-api-access-4hflb for pod openshift-network-diagnostics/network-check-target-b669x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:45:23.906078 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:23.906023    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb podName:7f4a7724-28ad-4bdb-938b-058b8e0bbcf5 nodeName:}" failed. No retries permitted until 2026-07-24 15:45:25.906004113 +0000 UTC m=+5.190331704 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4hflb" (UniqueName: "kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb") pod "network-check-target-b669x" (UID: "7f4a7724-28ad-4bdb-938b-058b8e0bbcf5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:45:23.970642 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:23.970556    2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:45:24.127789 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:24.127733    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 15:40:22 +0000 UTC" deadline="2028-02-13 03:29:13.327004897 +0000 UTC"
Jul 24 15:45:24.127789 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:24.127784    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13643h43m49.199223212s"
Jul 24 15:45:24.218075 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:24.217519    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:24.218075 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:24.217565    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:24.218075 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:24.217668    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:24.218075 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:24.217736    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:24.218075 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:24.217791    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:24.218075 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:24.217862    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:24.519681 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:24.519646    2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 24 15:45:25.724301 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:25.724200    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret\") pod \"global-pull-secret-syncer-k49df\" (UID: \"79ccf6fc-b5a8-4aff-bcb2-c14898719ac4\") " pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:25.724730 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:25.724367    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:45:25.724730 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:25.724443    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret podName:79ccf6fc-b5a8-4aff-bcb2-c14898719ac4 nodeName:}" failed. No retries permitted until 2026-07-24 15:45:29.724425044 +0000 UTC m=+9.008752634 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret") pod "global-pull-secret-syncer-k49df" (UID: "79ccf6fc-b5a8-4aff-bcb2-c14898719ac4") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:45:25.825205 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:25.825172    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs\") pod \"network-metrics-daemon-b86md\" (UID: \"c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e\") " pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:25.825383 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:25.825322    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:45:25.825427 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:25.825403    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs podName:c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e nodeName:}" failed. No retries permitted until 2026-07-24 15:45:29.825381482 +0000 UTC m=+9.109709070 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs") pod "network-metrics-daemon-b86md" (UID: "c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:45:25.925845 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:25.925809    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hflb\" (UniqueName: \"kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb\") pod \"network-check-target-b669x\" (UID: \"7f4a7724-28ad-4bdb-938b-058b8e0bbcf5\") " pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:25.926051 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:25.926012    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:45:25.926051 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:25.926028    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:45:25.926051 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:25.926040    2572 projected.go:196] Error preparing data for projected volume kube-api-access-4hflb for pod openshift-network-diagnostics/network-check-target-b669x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:45:25.926232 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:25.926093    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb podName:7f4a7724-28ad-4bdb-938b-058b8e0bbcf5 nodeName:}" failed. No retries permitted until 2026-07-24 15:45:29.926077875 +0000 UTC m=+9.210405461 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4hflb" (UniqueName: "kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb") pod "network-check-target-b669x" (UID: "7f4a7724-28ad-4bdb-938b-058b8e0bbcf5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:45:26.218884 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:26.218037    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:26.218884 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:26.218170    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:26.218884 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:26.218574    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:26.218884 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:26.218681    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:26.218884 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:26.218726    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:26.218884 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:26.218787    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:28.217750 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:28.217715    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:28.217750 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:28.217732    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:28.218231 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:28.217736    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:28.218231 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:28.217828    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:28.218231 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:28.217947    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:28.218231 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:28.218023    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:29.756041 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:29.755945    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret\") pod \"global-pull-secret-syncer-k49df\" (UID: \"79ccf6fc-b5a8-4aff-bcb2-c14898719ac4\") " pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:29.756483 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:29.756095    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:45:29.756483 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:29.756169    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret podName:79ccf6fc-b5a8-4aff-bcb2-c14898719ac4 nodeName:}" failed. No retries permitted until 2026-07-24 15:45:37.756148116 +0000 UTC m=+17.040475705 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret") pod "global-pull-secret-syncer-k49df" (UID: "79ccf6fc-b5a8-4aff-bcb2-c14898719ac4") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:45:29.856639 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:29.856597    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs\") pod \"network-metrics-daemon-b86md\" (UID: \"c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e\") " pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:29.856833 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:29.856784    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:45:29.856890 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:29.856868    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs podName:c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e nodeName:}" failed. No retries permitted until 2026-07-24 15:45:37.856846261 +0000 UTC m=+17.141173846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs") pod "network-metrics-daemon-b86md" (UID: "c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:45:29.957449 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:29.957403    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hflb\" (UniqueName: \"kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb\") pod \"network-check-target-b669x\" (UID: \"7f4a7724-28ad-4bdb-938b-058b8e0bbcf5\") " pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:29.957622 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:29.957579    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:45:29.957622 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:29.957602    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:45:29.957622 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:29.957612    2572 projected.go:196] Error preparing data for projected volume kube-api-access-4hflb for pod openshift-network-diagnostics/network-check-target-b669x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:45:29.957734 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:29.957665    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb podName:7f4a7724-28ad-4bdb-938b-058b8e0bbcf5 nodeName:}" failed. No retries permitted until 2026-07-24 15:45:37.957651519 +0000 UTC m=+17.241979104 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4hflb" (UniqueName: "kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb") pod "network-check-target-b669x" (UID: "7f4a7724-28ad-4bdb-938b-058b8e0bbcf5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:45:30.218601 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:30.218011    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:30.218601 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:30.218037    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:30.218601 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:30.218026    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:30.218601 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:30.218176    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:30.218601 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:30.218250    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:30.218601 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:30.218322    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:32.217453 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:32.217412    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:32.217959 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:32.217576    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:32.217959 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:32.217623    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:32.217959 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:32.217631    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:32.217959 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:32.217743    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:32.217959 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:32.217810    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:34.217937 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:34.217881    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:34.218369 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:34.217881    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:34.218369 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:34.218046    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:34.218369 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:34.217881    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:34.218369 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:34.218089    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:34.218369 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:34.218167    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:36.217426 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:36.217395    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:36.217958 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:36.217442    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:36.217958 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:36.217507    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:36.217958 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:36.217522    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:36.217958 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:36.217590    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:36.217958 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:36.217669    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:36.418117 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:36.418080    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ng2nc" event={"ID":"3caca464-55f2-4c4d-8773-216a1d9212f6","Type":"ContainerStarted","Data":"f5778032e254d48265d72a2d0f6abf488b4a093be72058cd8f63d3e3cb0e1a21"}
Jul 24 15:45:36.421745 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:36.421714    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-124.ec2.internal" event={"ID":"d00c243b91d89928c6de8f1bbae37e8e","Type":"ContainerStarted","Data":"15af679b3490a6c5115e1c7e53d2b7a041e589a37d9ca3b6e0d63fe50acd8020"}
Jul 24 15:45:36.423424 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:36.423350    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w6q8q" event={"ID":"f00bbdbf-d8c1-482d-890b-4917961038a5","Type":"ContainerStarted","Data":"71916dbffe12cb8c63741858618aefaa5313466f3816bfb3a54daf8bd74a38ef"}
Jul 24 15:45:36.434345 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:36.434294    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ng2nc" podStartSLOduration=2.47697054 podStartE2EDuration="15.43427772s" totalImagesPullingTime="12.95730718s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:45:36.433965887 +0000 UTC m=+15.718293501" watchObservedRunningTime="2026-07-24 15:45:36.43427772 +0000 UTC m=+15.718605328"
Jul 24 15:45:36.447306 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:36.447242    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-124.ec2.internal" podStartSLOduration=14.447220247 podStartE2EDuration="14.447220247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:45:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:45:36.446630053 +0000 UTC m=+15.730957660" watchObservedRunningTime="2026-07-24 15:45:36.447220247 +0000 UTC m=+15.731547913"
Jul 24 15:45:36.470452 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:36.470386    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-w6q8q" podStartSLOduration=1.956670913 podStartE2EDuration="15.470362272s" totalImagesPullingTime="13.513691359s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:45:36.468560385 +0000 UTC m=+15.752888024" watchObservedRunningTime="2026-07-24 15:45:36.470362272 +0000 UTC m=+15.754689879"
Jul 24 15:45:37.427540 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:37.427295    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w6mr8" event={"ID":"50c1abb8-1abe-4f3d-8dea-7dd210b396ee","Type":"ContainerStarted","Data":"aa2cd2e4d36411909f32fbf11520f52bbe29ba717cbb1da6506eab0d2e542f45"}
Jul 24 15:45:37.429102 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:37.429070    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jz8pt" event={"ID":"ce828380-8975-40e2-b6ae-093b77698dbe","Type":"ContainerStarted","Data":"133ac96155432e0df6ac343046364e14d705686059c03b3e2261a98ebee73718"}
Jul 24 15:45:37.430753 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:37.430722    2572 generic.go:356] "Generic (PLEG): container finished" podID="46f61359-1f52-4ef6-9831-b55a3e3096e5" containerID="89dca2dc1f2b730ffb82fc97b0c3c253cb8e3bcd39669560042e9e11ba45831b" exitCode=0
Jul 24 15:45:37.430869 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:37.430778    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ck8qc" event={"ID":"46f61359-1f52-4ef6-9831-b55a3e3096e5","Type":"ContainerDied","Data":"89dca2dc1f2b730ffb82fc97b0c3c253cb8e3bcd39669560042e9e11ba45831b"}
Jul 24 15:45:37.432398 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:37.432309    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4lpwk" event={"ID":"ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94","Type":"ContainerStarted","Data":"ea476a77bac295926d3f4855b42272704622113701c7b48ac14f69568fd3a358"}
Jul 24 15:45:37.434210 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:37.433744    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5xwjc" event={"ID":"f6f39b6e-359d-4a0a-9b1e-4b1afa097802","Type":"ContainerStarted","Data":"dded43b8b655aaed6d9dac4dd1699390332a858b35a9a952249289f954b9555d"}
Jul 24 15:45:37.435276 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:37.435250    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm" event={"ID":"c86da29a-9f5b-4763-82eb-3f9569146b2e","Type":"ContainerStarted","Data":"6dccddf901515625af53bd9b857f90e5ce401cabd3663a6e209b07f442328b40"}
Jul 24 15:45:37.437078 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:37.436865    2572 generic.go:356] "Generic (PLEG): container finished" podID="cf09905fff65fcdc788ab624bd938c38" containerID="a333dece47903ca2db61c9e18cdc68aa65ac292798b1c70f9ef228040b7d3541" exitCode=0
Jul 24 15:45:37.437078 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:37.436914    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal" event={"ID":"cf09905fff65fcdc788ab624bd938c38","Type":"ContainerDied","Data":"a333dece47903ca2db61c9e18cdc68aa65ac292798b1c70f9ef228040b7d3541"}
Jul 24 15:45:37.445880 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:37.445834    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-w6mr8" podStartSLOduration=3.517515624 podStartE2EDuration="16.445818648s" totalImagesPullingTime="12.928303024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:45:37.445310536 +0000 UTC m=+16.729638143" watchObservedRunningTime="2026-07-24 15:45:37.445818648 +0000 UTC m=+16.730146254"
Jul 24 15:45:37.496028 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:37.495969    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4lpwk" podStartSLOduration=3.444487041 podStartE2EDuration="16.495950852s" totalImagesPullingTime="13.051463811s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:45:37.472457434 +0000 UTC m=+16.756785045" watchObservedRunningTime="2026-07-24 15:45:37.495950852 +0000 UTC m=+16.780278459"
Jul 24 15:45:37.512843 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:37.512781    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5xwjc" podStartSLOduration=3.459226716 podStartE2EDuration="16.51275954s" totalImagesPullingTime="13.053532824s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:45:37.512032041 +0000 UTC m=+16.796359647" watchObservedRunningTime="2026-07-24 15:45:37.51275954 +0000 UTC m=+16.797087148"
Jul 24 15:45:37.816881 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:37.816852    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret\") pod \"global-pull-secret-syncer-k49df\" (UID: \"79ccf6fc-b5a8-4aff-bcb2-c14898719ac4\") " pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:37.817088 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:37.817052    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:45:37.817151 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:37.817118    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret podName:79ccf6fc-b5a8-4aff-bcb2-c14898719ac4 nodeName:}" failed. No retries permitted until 2026-07-24 15:45:53.817097772 +0000 UTC m=+33.101425378 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret") pod "global-pull-secret-syncer-k49df" (UID: "79ccf6fc-b5a8-4aff-bcb2-c14898719ac4") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:45:37.817412 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:37.817394    2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 24 15:45:37.917778 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:37.917740    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs\") pod \"network-metrics-daemon-b86md\" (UID: \"c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e\") " pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:37.917999 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:37.917880    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:45:37.917999 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:37.917966    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs podName:c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e nodeName:}" failed. No retries permitted until 2026-07-24 15:45:53.917945077 +0000 UTC m=+33.202272671 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs") pod "network-metrics-daemon-b86md" (UID: "c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:45:38.018684 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:38.018600    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hflb\" (UniqueName: \"kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb\") pod \"network-check-target-b669x\" (UID: \"7f4a7724-28ad-4bdb-938b-058b8e0bbcf5\") " pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:38.018832 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:38.018803    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:45:38.018832 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:38.018830    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:45:38.018906 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:38.018839    2572 projected.go:196] Error preparing data for projected volume kube-api-access-4hflb for pod openshift-network-diagnostics/network-check-target-b669x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:45:38.018906 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:38.018887    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb podName:7f4a7724-28ad-4bdb-938b-058b8e0bbcf5 nodeName:}" failed. No retries permitted until 2026-07-24 15:45:54.018871776 +0000 UTC m=+33.303199361 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4hflb" (UniqueName: "kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb") pod "network-check-target-b669x" (UID: "7f4a7724-28ad-4bdb-938b-058b8e0bbcf5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:45:38.149877 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:38.149756    2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T15:45:37.817410788Z","UUID":"b1726865-0de6-4884-a9f5-0585b1ffcb5d","Handler":null,"Name":"","Endpoint":""}
Jul 24 15:45:38.152321 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:38.152284    2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 24 15:45:38.152321 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:38.152328    2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 24 15:45:38.218266 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:38.218230    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:38.218454 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:38.218274    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:38.218454 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:38.218246    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:38.218454 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:38.218436    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:38.218607 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:38.218560    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:38.218663 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:38.218621    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:38.444466 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:38.444432    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm" event={"ID":"c86da29a-9f5b-4763-82eb-3f9569146b2e","Type":"ContainerStarted","Data":"ba6326290c3c7bd725b818b462f22499c97ee13876abcd8c2910181d13cc20d4"}
Jul 24 15:45:38.447508 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:38.447481    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal" event={"ID":"cf09905fff65fcdc788ab624bd938c38","Type":"ContainerStarted","Data":"cdfb2245c5ed24a189c3b8baebe87932ff80da77a5a4de76367e4767168a6be4"}
Jul 24 15:45:38.466059 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:38.465719    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jz8pt" podStartSLOduration=4.4634746 podStartE2EDuration="17.465699024s" totalImagesPullingTime="13.002224424s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:45:37.526694012 +0000 UTC m=+16.811021619" watchObservedRunningTime="2026-07-24 15:45:38.465699024 +0000 UTC m=+17.750026632"
Jul 24 15:45:40.218140 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:40.217841    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:40.218140 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:40.217848    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:40.218635 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:40.218143    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:40.218635 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:40.217848    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:40.218635 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:40.218209    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:40.218635 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:40.218260    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:40.455095 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:40.455055    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm" event={"ID":"c86da29a-9f5b-4763-82eb-3f9569146b2e","Type":"ContainerStarted","Data":"18e2b6531ac2b1c0f4d914ec1a3e35034b697da7e7dddeef679b93c3cb905fac"}
Jul 24 15:45:40.473140 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:40.473036    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4tlzm" podStartSLOduration=3.277034678 podStartE2EDuration="19.473019398s" totalImagesPullingTime="16.19598472s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:45:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:45:40.472713709 +0000 UTC m=+19.757041307" watchObservedRunningTime="2026-07-24 15:45:40.473019398 +0000 UTC m=+19.757347005"
Jul 24 15:45:40.473708 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:40.473669    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-124.ec2.internal" podStartSLOduration=18.473658059 podStartE2EDuration="18.473658059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:45:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:45:38.46603359 +0000 UTC m=+17.750361198" watchObservedRunningTime="2026-07-24 15:45:40.473658059 +0000 UTC m=+19.757985678"
Jul 24 15:45:41.536504 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:41.536462    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-w6mr8"
Jul 24 15:45:41.537121 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:41.537098    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-w6mr8"
Jul 24 15:45:42.217832 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:42.217797    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:42.218052 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:42.217841    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:42.218052 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:42.217815    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:42.218052 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:42.217913    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:42.218052 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:42.218001    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:42.218237 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:42.218105    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:43.465202 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:43.465096    2572 generic.go:356] "Generic (PLEG): container finished" podID="46f61359-1f52-4ef6-9831-b55a3e3096e5" containerID="d139727242a7f238ca3b362730f23d165492d4e3eefb60c4911cde7d34a8aa95" exitCode=0
Jul 24 15:45:43.465705 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:43.465193    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ck8qc" event={"ID":"46f61359-1f52-4ef6-9831-b55a3e3096e5","Type":"ContainerDied","Data":"d139727242a7f238ca3b362730f23d165492d4e3eefb60c4911cde7d34a8aa95"}
Jul 24 15:45:44.217801 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:44.217759    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:44.217801 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:44.217806    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:44.218080 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:44.217873    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:44.218080 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:44.217891    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:44.218080 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:44.217978    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:44.218080 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:44.218029    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:46.112388 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:46.112353    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-w6mr8"
Jul 24 15:45:46.112858 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:46.112511    2572 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 24 15:45:46.113038 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:46.113014    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-w6mr8"
Jul 24 15:45:46.217782 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:46.217743    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:46.217782 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:46.217780    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:46.218026 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:46.217744    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:46.218026 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:46.217871    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:46.218026 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:46.217960    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:46.218159 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:46.218046    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:48.217607 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:48.217569    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:48.217607 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:48.217598    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:48.218096 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:48.217598    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:48.218096 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:48.217689    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:48.218096 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:48.217767    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:48.218096 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:48.217836    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:49.480228 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:49.480195    2572 generic.go:356] "Generic (PLEG): container finished" podID="46f61359-1f52-4ef6-9831-b55a3e3096e5" containerID="174f66b0c4bdc137f33c178e7f6a3c2d7a3ed9506e4695e2dec5363b9d34980c" exitCode=0
Jul 24 15:45:49.480828 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:49.480282    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ck8qc" event={"ID":"46f61359-1f52-4ef6-9831-b55a3e3096e5","Type":"ContainerDied","Data":"174f66b0c4bdc137f33c178e7f6a3c2d7a3ed9506e4695e2dec5363b9d34980c"}
Jul 24 15:45:49.483148 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:49.483130    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 15:45:49.483535 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:49.483511    2572 generic.go:356] "Generic (PLEG): container finished" podID="0e30eb70-1243-4f4b-8a15-5df72a224f51" containerID="438f6f61c82bad08d0b413e5b9d111b14ce1f0d98a5fd843a373dc4727182bf3" exitCode=1
Jul 24 15:45:49.483646 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:49.483548    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js" event={"ID":"0e30eb70-1243-4f4b-8a15-5df72a224f51","Type":"ContainerStarted","Data":"ad4e2c1204411b13c118838fd760024c9ca03638240bccca5d8bf779641eb153"}
Jul 24 15:45:49.483646 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:49.483627    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js" event={"ID":"0e30eb70-1243-4f4b-8a15-5df72a224f51","Type":"ContainerStarted","Data":"6edbc4908dee2213e0ef14cc807c9733c5932d404b73998a032e5b525a61e381"}
Jul 24 15:45:49.483725 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:49.483677    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js" event={"ID":"0e30eb70-1243-4f4b-8a15-5df72a224f51","Type":"ContainerStarted","Data":"e20914e27dd976785187a91da96e850473d1cfe88f113ea3c0c8d0706f8372df"}
Jul 24 15:45:49.483758 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:49.483725    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js" event={"ID":"0e30eb70-1243-4f4b-8a15-5df72a224f51","Type":"ContainerStarted","Data":"a9a33ddb265c0c997ce6f5a5b83c5dfde0364e77528d189646ef1a4b2a7662cf"}
Jul 24 15:45:49.483789 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:49.483768    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js" event={"ID":"0e30eb70-1243-4f4b-8a15-5df72a224f51","Type":"ContainerDied","Data":"438f6f61c82bad08d0b413e5b9d111b14ce1f0d98a5fd843a373dc4727182bf3"}
Jul 24 15:45:49.483823 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:49.483814    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js" event={"ID":"0e30eb70-1243-4f4b-8a15-5df72a224f51","Type":"ContainerStarted","Data":"bb442e9fa95c93434d53a5ea3da137a90aaa6143d1ce174ea1673f3acf3e7850"}
Jul 24 15:45:50.217440 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:50.217398    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:50.217715 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:50.217401    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:50.217715 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:50.217526    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:50.217715 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:50.217631    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:50.217715 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:50.217413    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:50.217912 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:50.217714    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:51.490246 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:51.490208    2572 generic.go:356] "Generic (PLEG): container finished" podID="46f61359-1f52-4ef6-9831-b55a3e3096e5" containerID="7e8eca168185ccb31a361b04a7cb90d2c0ca73b286079df05096957388e955d0" exitCode=0
Jul 24 15:45:51.490794 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:51.490297    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ck8qc" event={"ID":"46f61359-1f52-4ef6-9831-b55a3e3096e5","Type":"ContainerDied","Data":"7e8eca168185ccb31a361b04a7cb90d2c0ca73b286079df05096957388e955d0"}
Jul 24 15:45:51.493672 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:51.493654    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 15:45:51.494029 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:51.494009    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js" event={"ID":"0e30eb70-1243-4f4b-8a15-5df72a224f51","Type":"ContainerStarted","Data":"9c001d7f8c616362b5634d3d2aebb403a4763bf6e27a8cb9638d68e86b55c05e"}
Jul 24 15:45:52.218192 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:52.218159    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:52.218379 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:52.218159    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:52.218379 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:52.218308    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:52.218489 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:52.218381    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:52.218489 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:52.218160    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:52.218489 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:52.218475    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:53.837435 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:53.837147    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret\") pod \"global-pull-secret-syncer-k49df\" (UID: \"79ccf6fc-b5a8-4aff-bcb2-c14898719ac4\") " pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:53.837435 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:53.837269    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 24 15:45:53.837912 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:53.837484    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret podName:79ccf6fc-b5a8-4aff-bcb2-c14898719ac4 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:25.837456711 +0000 UTC m=+65.121784300 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret") pod "global-pull-secret-syncer-k49df" (UID: "79ccf6fc-b5a8-4aff-bcb2-c14898719ac4") : object "kube-system"/"original-pull-secret" not registered
Jul 24 15:45:53.937982 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:53.937948    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs\") pod \"network-metrics-daemon-b86md\" (UID: \"c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e\") " pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:53.938140 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:53.938108    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:45:53.938179 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:53.938171    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs podName:c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e nodeName:}" failed. No retries permitted until 2026-07-24 15:46:25.938153282 +0000 UTC m=+65.222480880 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs") pod "network-metrics-daemon-b86md" (UID: "c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 24 15:45:54.038817 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:54.038777    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hflb\" (UniqueName: \"kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb\") pod \"network-check-target-b669x\" (UID: \"7f4a7724-28ad-4bdb-938b-058b8e0bbcf5\") " pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:54.039026 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:54.038971    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 24 15:45:54.039026 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:54.038991    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 24 15:45:54.039026 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:54.039002    2572 projected.go:196] Error preparing data for projected volume kube-api-access-4hflb for pod openshift-network-diagnostics/network-check-target-b669x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:45:54.039135 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:54.039051    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb podName:7f4a7724-28ad-4bdb-938b-058b8e0bbcf5 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:26.039036771 +0000 UTC m=+65.323364355 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4hflb" (UniqueName: "kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb") pod "network-check-target-b669x" (UID: "7f4a7724-28ad-4bdb-938b-058b8e0bbcf5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 24 15:45:54.218075 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:54.218047    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:54.218245 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:54.218047    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:54.218245 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:54.218150    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:54.218245 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:54.218047    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:54.218245 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:54.218231    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:54.218374 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:54.218293    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:54.505217 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:54.505148    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 15:45:54.505617 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:54.505582    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js" event={"ID":"0e30eb70-1243-4f4b-8a15-5df72a224f51","Type":"ContainerStarted","Data":"086a2b8d7eb1eae2e2c0c9679b540db9ac0522088419c89025e14e7758ffe2f5"}
Jul 24 15:45:54.505867 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:54.505840    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:54.506011 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:54.505949    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:54.506011 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:54.505998    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:54.506116 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:54.506070    2572 scope.go:117] "RemoveContainer" containerID="438f6f61c82bad08d0b413e5b9d111b14ce1f0d98a5fd843a373dc4727182bf3"
Jul 24 15:45:54.525649 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:54.525625    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:54.525819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:54.525806    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:45:55.513379 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:55.513347    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 15:45:55.514299 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:55.514272    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js" event={"ID":"0e30eb70-1243-4f4b-8a15-5df72a224f51","Type":"ContainerStarted","Data":"13af09a327ee43a6d9de985882c62c27765eaeeb90d823401d494b87c78d78b3"}
Jul 24 15:45:55.546528 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:55.546479    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js" podStartSLOduration=8.542769297 podStartE2EDuration="34.546462851s" totalImagesPullingTime="26.003693554s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=2 observedRunningTime="2026-07-24 15:45:55.545495494 +0000 UTC m=+34.829823102" watchObservedRunningTime="2026-07-24 15:45:55.546462851 +0000 UTC m=+34.830790449"
Jul 24 15:45:55.653654 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:55.653083    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k49df"]
Jul 24 15:45:55.653654 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:55.653216    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:55.653654 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:55.653325    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:55.656845 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:55.656816    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b669x"]
Jul 24 15:45:55.657008 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:55.656944    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:55.657073 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:55.657036    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:55.657575 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:55.657550    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b86md"]
Jul 24 15:45:55.657673 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:55.657654    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:55.657771 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:55.657739    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:56.519804 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:56.519561    2572 generic.go:356] "Generic (PLEG): container finished" podID="46f61359-1f52-4ef6-9831-b55a3e3096e5" containerID="01645de2de5198ed909d21882e084102eb8c9f07cd81dfb9e6d076c8e86411cc" exitCode=0
Jul 24 15:45:56.519804 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:56.519642    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ck8qc" event={"ID":"46f61359-1f52-4ef6-9831-b55a3e3096e5","Type":"ContainerDied","Data":"01645de2de5198ed909d21882e084102eb8c9f07cd81dfb9e6d076c8e86411cc"}
Jul 24 15:45:57.218199 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:57.218163    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:57.218199 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:57.218182    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:57.218459 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:57.218164    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:57.218459 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:57.218277    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:57.218459 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:57.218363    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:57.218619 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:57.218458    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:45:57.526465 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:57.526378    2572 generic.go:356] "Generic (PLEG): container finished" podID="46f61359-1f52-4ef6-9831-b55a3e3096e5" containerID="bcc90b6b0da9de1af99cac6268a7e1daafeaa574f2a7009b7a7a06cb33a2c870" exitCode=0
Jul 24 15:45:57.526465 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:57.526424    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ck8qc" event={"ID":"46f61359-1f52-4ef6-9831-b55a3e3096e5","Type":"ContainerDied","Data":"bcc90b6b0da9de1af99cac6268a7e1daafeaa574f2a7009b7a7a06cb33a2c870"}
Jul 24 15:45:58.532477 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:58.532392    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ck8qc" event={"ID":"46f61359-1f52-4ef6-9831-b55a3e3096e5","Type":"ContainerStarted","Data":"16d1fb83f56e4a4af350428c878d4c2c8604b94e5028a294d593e5799bab95c0"}
Jul 24 15:45:58.557965 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:58.557893    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ck8qc" podStartSLOduration=7.991908261 podStartE2EDuration="37.55787755s" totalImagesPullingTime="29.565969289s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:45:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:45:58.557490687 +0000 UTC m=+37.841818293" watchObservedRunningTime="2026-07-24 15:45:58.55787755 +0000 UTC m=+37.842205157"
Jul 24 15:45:59.218161 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:59.218116    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:45:59.218161 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:59.218155    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:45:59.218369 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:45:59.218116    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:45:59.218369 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:59.218245    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:45:59.218369 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:59.218309    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:45:59.218464 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:45:59.218401    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:46:01.218819 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:01.218775    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:46:01.219346 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:01.218859    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:46:01.219346 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:01.218862    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b669x" podUID="7f4a7724-28ad-4bdb-938b-058b8e0bbcf5"
Jul 24 15:46:01.219346 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:01.218972    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b86md" podUID="c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e"
Jul 24 15:46:01.219346 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:01.219020    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:46:01.219346 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:01.219071    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k49df" podUID="79ccf6fc-b5a8-4aff-bcb2-c14898719ac4"
Jul 24 15:46:02.036705 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.036624    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-124.ec2.internal" event="NodeReady"
Jul 24 15:46:02.036831 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.036740    2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 24 15:46:02.085842 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.085814    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8wnvz"]
Jul 24 15:46:02.103375 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.103345    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5jtpt"]
Jul 24 15:46:02.103555 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.103436    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:02.106587 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.106563    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vwj8g\""
Jul 24 15:46:02.106587 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.106563    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 24 15:46:02.107162 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.107138    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 24 15:46:02.121610 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.121587    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8wnvz"]
Jul 24 15:46:02.121687 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.121619    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5jtpt"]
Jul 24 15:46:02.121687 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.121682    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5jtpt"
Jul 24 15:46:02.124641 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.124583    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 24 15:46:02.124641 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.124593    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-82l8s\""
Jul 24 15:46:02.124766 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.124589    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 24 15:46:02.124802 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.124791    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 24 15:46:02.197407 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.197368    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5746ff45-485c-498a-904e-8c506e3cffc4-tmp-dir\") pod \"dns-default-8wnvz\" (UID: \"5746ff45-485c-498a-904e-8c506e3cffc4\") " pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:02.197564 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.197436    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2rqx\" (UniqueName: \"kubernetes.io/projected/5746ff45-485c-498a-904e-8c506e3cffc4-kube-api-access-t2rqx\") pod \"dns-default-8wnvz\" (UID: \"5746ff45-485c-498a-904e-8c506e3cffc4\") " pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:02.197564 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.197464    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5746ff45-485c-498a-904e-8c506e3cffc4-config-volume\") pod \"dns-default-8wnvz\" (UID: \"5746ff45-485c-498a-904e-8c506e3cffc4\") " pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:02.197564 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.197489    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls\") pod \"dns-default-8wnvz\" (UID: \"5746ff45-485c-498a-904e-8c506e3cffc4\") " pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:02.298620 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.298534    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acb09411-3411-4d23-875c-2b4768a9a968-cert\") pod \"ingress-canary-5jtpt\" (UID: \"acb09411-3411-4d23-875c-2b4768a9a968\") " pod="openshift-ingress-canary/ingress-canary-5jtpt"
Jul 24 15:46:02.298620 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.298588    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2rqx\" (UniqueName: \"kubernetes.io/projected/5746ff45-485c-498a-904e-8c506e3cffc4-kube-api-access-t2rqx\") pod \"dns-default-8wnvz\" (UID: \"5746ff45-485c-498a-904e-8c506e3cffc4\") " pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:02.299280 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.298615    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl2sf\" (UniqueName: \"kubernetes.io/projected/acb09411-3411-4d23-875c-2b4768a9a968-kube-api-access-tl2sf\") pod \"ingress-canary-5jtpt\" (UID: \"acb09411-3411-4d23-875c-2b4768a9a968\") " pod="openshift-ingress-canary/ingress-canary-5jtpt"
Jul 24 15:46:02.299280 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.298692    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5746ff45-485c-498a-904e-8c506e3cffc4-config-volume\") pod \"dns-default-8wnvz\" (UID: \"5746ff45-485c-498a-904e-8c506e3cffc4\") " pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:02.299280 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.298739    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls\") pod \"dns-default-8wnvz\" (UID: \"5746ff45-485c-498a-904e-8c506e3cffc4\") " pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:02.299280 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.298838    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5746ff45-485c-498a-904e-8c506e3cffc4-tmp-dir\") pod \"dns-default-8wnvz\" (UID: \"5746ff45-485c-498a-904e-8c506e3cffc4\") " pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:02.299280 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:02.298859    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:46:02.299280 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:02.298942    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls podName:5746ff45-485c-498a-904e-8c506e3cffc4 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:02.798905357 +0000 UTC m=+42.083232956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls") pod "dns-default-8wnvz" (UID: "5746ff45-485c-498a-904e-8c506e3cffc4") : secret "dns-default-metrics-tls" not found
Jul 24 15:46:02.299280 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.299125    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5746ff45-485c-498a-904e-8c506e3cffc4-tmp-dir\") pod \"dns-default-8wnvz\" (UID: \"5746ff45-485c-498a-904e-8c506e3cffc4\") " pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:02.299485 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.299338    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5746ff45-485c-498a-904e-8c506e3cffc4-config-volume\") pod \"dns-default-8wnvz\" (UID: \"5746ff45-485c-498a-904e-8c506e3cffc4\") " pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:02.311280 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.311258    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2rqx\" (UniqueName: \"kubernetes.io/projected/5746ff45-485c-498a-904e-8c506e3cffc4-kube-api-access-t2rqx\") pod \"dns-default-8wnvz\" (UID: \"5746ff45-485c-498a-904e-8c506e3cffc4\") " pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:02.400044 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.400013    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acb09411-3411-4d23-875c-2b4768a9a968-cert\") pod \"ingress-canary-5jtpt\" (UID: \"acb09411-3411-4d23-875c-2b4768a9a968\") " pod="openshift-ingress-canary/ingress-canary-5jtpt"
Jul 24 15:46:02.400210 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.400060    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl2sf\" (UniqueName: \"kubernetes.io/projected/acb09411-3411-4d23-875c-2b4768a9a968-kube-api-access-tl2sf\") pod \"ingress-canary-5jtpt\" (UID: \"acb09411-3411-4d23-875c-2b4768a9a968\") " pod="openshift-ingress-canary/ingress-canary-5jtpt"
Jul 24 15:46:02.402583 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.402553    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acb09411-3411-4d23-875c-2b4768a9a968-cert\") pod \"ingress-canary-5jtpt\" (UID: \"acb09411-3411-4d23-875c-2b4768a9a968\") " pod="openshift-ingress-canary/ingress-canary-5jtpt"
Jul 24 15:46:02.410613 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.410590    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl2sf\" (UniqueName: \"kubernetes.io/projected/acb09411-3411-4d23-875c-2b4768a9a968-kube-api-access-tl2sf\") pod \"ingress-canary-5jtpt\" (UID: \"acb09411-3411-4d23-875c-2b4768a9a968\") " pod="openshift-ingress-canary/ingress-canary-5jtpt"
Jul 24 15:46:02.430224 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.430189    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5jtpt"
Jul 24 15:46:02.628637 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.628473    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5jtpt"]
Jul 24 15:46:02.631776 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:46:02.631753    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacb09411_3411_4d23_875c_2b4768a9a968.slice/crio-9887a0777f944ce9950260d9426c833df1e564e5757b4f23ab364a5d527f64ad WatchSource:0}: Error finding container 9887a0777f944ce9950260d9426c833df1e564e5757b4f23ab364a5d527f64ad: Status 404 returned error can't find the container with id 9887a0777f944ce9950260d9426c833df1e564e5757b4f23ab364a5d527f64ad
Jul 24 15:46:02.802404 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:02.802367    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls\") pod \"dns-default-8wnvz\" (UID: \"5746ff45-485c-498a-904e-8c506e3cffc4\") " pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:02.802580 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:02.802495    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:46:02.802580 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:02.802564    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls podName:5746ff45-485c-498a-904e-8c506e3cffc4 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:03.802536087 +0000 UTC m=+43.086863674 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls") pod "dns-default-8wnvz" (UID: "5746ff45-485c-498a-904e-8c506e3cffc4") : secret "dns-default-metrics-tls" not found
Jul 24 15:46:03.218104 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:03.218070    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:46:03.218284 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:03.218068    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:46:03.218358 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:03.218070    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:46:03.221640 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:03.221615    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 15:46:03.222689 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:03.222669    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 15:46:03.222689 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:03.222686    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 15:46:03.223290 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:03.223202    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pqxfk\""
Jul 24 15:46:03.226276 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:03.223486    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cbp7q\""
Jul 24 15:46:03.226276 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:03.223549    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 15:46:03.545413 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:03.545335    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5jtpt" event={"ID":"acb09411-3411-4d23-875c-2b4768a9a968","Type":"ContainerStarted","Data":"9887a0777f944ce9950260d9426c833df1e564e5757b4f23ab364a5d527f64ad"}
Jul 24 15:46:03.810150 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:03.810058    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls\") pod \"dns-default-8wnvz\" (UID: \"5746ff45-485c-498a-904e-8c506e3cffc4\") " pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:03.810288 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:03.810193    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:46:03.810288 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:03.810251    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls podName:5746ff45-485c-498a-904e-8c506e3cffc4 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:05.810236517 +0000 UTC m=+45.094564104 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls") pod "dns-default-8wnvz" (UID: "5746ff45-485c-498a-904e-8c506e3cffc4") : secret "dns-default-metrics-tls" not found
Jul 24 15:46:05.826590 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:05.826535    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls\") pod \"dns-default-8wnvz\" (UID: \"5746ff45-485c-498a-904e-8c506e3cffc4\") " pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:05.826975 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:05.826694    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:46:05.826975 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:05.826765    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls podName:5746ff45-485c-498a-904e-8c506e3cffc4 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:09.82674644 +0000 UTC m=+49.111074024 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls") pod "dns-default-8wnvz" (UID: "5746ff45-485c-498a-904e-8c506e3cffc4") : secret "dns-default-metrics-tls" not found
Jul 24 15:46:06.554118 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:06.554087    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5jtpt" event={"ID":"acb09411-3411-4d23-875c-2b4768a9a968","Type":"ContainerStarted","Data":"c2adc856f60a7429a95eb5a42f7f963b50e91fd5ae7c903ed5444aee1b29e682"}
Jul 24 15:46:06.571858 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:06.571812    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5jtpt" podStartSLOduration=1.536106662 podStartE2EDuration="4.571796045s" totalImagesPullingTime="3.035689383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:46:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:46:06.570777608 +0000 UTC m=+45.855105214" watchObservedRunningTime="2026-07-24 15:46:06.571796045 +0000 UTC m=+45.856123652"
Jul 24 15:46:09.855420 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:09.855364    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls\") pod \"dns-default-8wnvz\" (UID: \"5746ff45-485c-498a-904e-8c506e3cffc4\") " pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:09.855778 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:09.855500    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:46:09.855778 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:09.855554    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls podName:5746ff45-485c-498a-904e-8c506e3cffc4 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:17.855538759 +0000 UTC m=+57.139866352 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls") pod "dns-default-8wnvz" (UID: "5746ff45-485c-498a-904e-8c506e3cffc4") : secret "dns-default-metrics-tls" not found
Jul 24 15:46:11.540841 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.540802    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-v8c2j"]
Jul 24 15:46:11.543804 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.543787    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-v8c2j"
Jul 24 15:46:11.546715 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.546681    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 24 15:46:11.547780 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.547763    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-jxlmf\""
Jul 24 15:46:11.547845 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.547802    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 24 15:46:11.563712 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.563683    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-v8c2j"]
Jul 24 15:46:11.592991 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.592956    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-z79q4"]
Jul 24 15:46:11.595865 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.595841    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:11.599229 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.599206    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vfsm2\""
Jul 24 15:46:11.599467 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.599449    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 24 15:46:11.600080 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.600062    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 24 15:46:11.600080 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.600068    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 24 15:46:11.600243 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.600076    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 24 15:46:11.614181 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.614161    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-z79q4"]
Jul 24 15:46:11.668353 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.668326    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw9p4\" (UniqueName: \"kubernetes.io/projected/568f5f8a-4f51-4104-9d52-f9495186f9ad-kube-api-access-vw9p4\") pod \"migrator-6865c9967-v8c2j\" (UID: \"568f5f8a-4f51-4104-9d52-f9495186f9ad\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-v8c2j"
Jul 24 15:46:11.769282 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.769243    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba905f82-a2c3-4b61-b253-d7a66300d7c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:11.769389 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.769302    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ba905f82-a2c3-4b61-b253-d7a66300d7c2-crio-socket\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:11.769389 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.769319    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba905f82-a2c3-4b61-b253-d7a66300d7c2-data-volume\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:11.769454 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.769392    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw9p4\" (UniqueName: \"kubernetes.io/projected/568f5f8a-4f51-4104-9d52-f9495186f9ad-kube-api-access-vw9p4\") pod \"migrator-6865c9967-v8c2j\" (UID: \"568f5f8a-4f51-4104-9d52-f9495186f9ad\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-v8c2j"
Jul 24 15:46:11.769454 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.769425    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:11.769454 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.769446    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jts24\" (UniqueName: \"kubernetes.io/projected/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-api-access-jts24\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:11.779974 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.779942    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw9p4\" (UniqueName: \"kubernetes.io/projected/568f5f8a-4f51-4104-9d52-f9495186f9ad-kube-api-access-vw9p4\") pod \"migrator-6865c9967-v8c2j\" (UID: \"568f5f8a-4f51-4104-9d52-f9495186f9ad\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-v8c2j"
Jul 24 15:46:11.852623 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.852545    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-v8c2j"
Jul 24 15:46:11.870641 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.870616    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ba905f82-a2c3-4b61-b253-d7a66300d7c2-crio-socket\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:11.870753 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.870645    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba905f82-a2c3-4b61-b253-d7a66300d7c2-data-volume\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:11.870753 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.870665    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:11.870753 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:11.870736    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:46:11.870753 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.870740    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jts24\" (UniqueName: \"kubernetes.io/projected/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-api-access-jts24\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:11.870974 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.870824    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ba905f82-a2c3-4b61-b253-d7a66300d7c2-crio-socket\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:11.870974 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.870851    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba905f82-a2c3-4b61-b253-d7a66300d7c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:11.870974 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:11.870878    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-rbac-proxy-cm podName:ba905f82-a2c3-4b61-b253-d7a66300d7c2 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:12.37085376 +0000 UTC m=+51.655181352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z79q4" (UID: "ba905f82-a2c3-4b61-b253-d7a66300d7c2") : configmap "kube-rbac-proxy" not found
Jul 24 15:46:11.870974 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:11.870948    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:46:11.871182 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:11.870999    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba905f82-a2c3-4b61-b253-d7a66300d7c2-insights-runtime-extractor-tls podName:ba905f82-a2c3-4b61-b253-d7a66300d7c2 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:12.370985085 +0000 UTC m=+51.655312672 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ba905f82-a2c3-4b61-b253-d7a66300d7c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z79q4" (UID: "ba905f82-a2c3-4b61-b253-d7a66300d7c2") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:46:11.871182 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.871003    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba905f82-a2c3-4b61-b253-d7a66300d7c2-data-volume\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:11.878815 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.878794    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jts24\" (UniqueName: \"kubernetes.io/projected/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-api-access-jts24\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:11.996452 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:11.996421    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-v8c2j"]
Jul 24 15:46:11.999477 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:46:11.999446    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod568f5f8a_4f51_4104_9d52_f9495186f9ad.slice/crio-c0d9b06060b4333ced5ad1895f093c3a6207a80a1b591a719e52faf27c079826 WatchSource:0}: Error finding container c0d9b06060b4333ced5ad1895f093c3a6207a80a1b591a719e52faf27c079826: Status 404 returned error can't find the container with id c0d9b06060b4333ced5ad1895f093c3a6207a80a1b591a719e52faf27c079826
Jul 24 15:46:12.374287 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:12.374251    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba905f82-a2c3-4b61-b253-d7a66300d7c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:12.374469 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:12.374307    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:12.374469 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:12.374378    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:46:12.374469 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:12.374390    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:46:12.374469 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:12.374444    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-rbac-proxy-cm podName:ba905f82-a2c3-4b61-b253-d7a66300d7c2 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:13.374431051 +0000 UTC m=+52.658758637 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z79q4" (UID: "ba905f82-a2c3-4b61-b253-d7a66300d7c2") : configmap "kube-rbac-proxy" not found
Jul 24 15:46:12.374469 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:12.374457    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba905f82-a2c3-4b61-b253-d7a66300d7c2-insights-runtime-extractor-tls podName:ba905f82-a2c3-4b61-b253-d7a66300d7c2 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:13.374450891 +0000 UTC m=+52.658778477 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ba905f82-a2c3-4b61-b253-d7a66300d7c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z79q4" (UID: "ba905f82-a2c3-4b61-b253-d7a66300d7c2") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:46:12.571220 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:12.571177    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-v8c2j" event={"ID":"568f5f8a-4f51-4104-9d52-f9495186f9ad","Type":"ContainerStarted","Data":"c0d9b06060b4333ced5ad1895f093c3a6207a80a1b591a719e52faf27c079826"}
Jul 24 15:46:13.272458 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.272428    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-p74fm"]
Jul 24 15:46:13.275309 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.275286    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-p74fm"
Jul 24 15:46:13.278128 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.278105    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-rjq94\""
Jul 24 15:46:13.278245 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.278108    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 24 15:46:13.279075 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.279058    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 24 15:46:13.279348 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.279322    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 24 15:46:13.279427 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.279372    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 24 15:46:13.287609 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.287542    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-p74fm"]
Jul 24 15:46:13.366627 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.366586    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5xwjc_f6f39b6e-359d-4a0a-9b1e-4b1afa097802/dns-node-resolver/0.log"
Jul 24 15:46:13.383870 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.383842    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/985ed3e7-90a9-4a31-853e-255f855d58d7-signing-cabundle\") pod \"service-ca-658d7b7c78-p74fm\" (UID: \"985ed3e7-90a9-4a31-853e-255f855d58d7\") " pod="openshift-service-ca/service-ca-658d7b7c78-p74fm"
Jul 24 15:46:13.384013 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.383893    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba905f82-a2c3-4b61-b253-d7a66300d7c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:13.384013 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.383984    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/985ed3e7-90a9-4a31-853e-255f855d58d7-signing-key\") pod \"service-ca-658d7b7c78-p74fm\" (UID: \"985ed3e7-90a9-4a31-853e-255f855d58d7\") " pod="openshift-service-ca/service-ca-658d7b7c78-p74fm"
Jul 24 15:46:13.384013 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:13.384006    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:46:13.384013 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.384007    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf66m\" (UniqueName: \"kubernetes.io/projected/985ed3e7-90a9-4a31-853e-255f855d58d7-kube-api-access-sf66m\") pod \"service-ca-658d7b7c78-p74fm\" (UID: \"985ed3e7-90a9-4a31-853e-255f855d58d7\") " pod="openshift-service-ca/service-ca-658d7b7c78-p74fm"
Jul 24 15:46:13.384143 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.384031    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:13.384143 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:13.384050    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba905f82-a2c3-4b61-b253-d7a66300d7c2-insights-runtime-extractor-tls podName:ba905f82-a2c3-4b61-b253-d7a66300d7c2 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:15.384037247 +0000 UTC m=+54.668364832 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ba905f82-a2c3-4b61-b253-d7a66300d7c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z79q4" (UID: "ba905f82-a2c3-4b61-b253-d7a66300d7c2") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:46:13.384143 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:13.384077    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:46:13.384243 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:13.384152    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-rbac-proxy-cm podName:ba905f82-a2c3-4b61-b253-d7a66300d7c2 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:15.384137293 +0000 UTC m=+54.668464879 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z79q4" (UID: "ba905f82-a2c3-4b61-b253-d7a66300d7c2") : configmap "kube-rbac-proxy" not found
Jul 24 15:46:13.484522 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.484493    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/985ed3e7-90a9-4a31-853e-255f855d58d7-signing-key\") pod \"service-ca-658d7b7c78-p74fm\" (UID: \"985ed3e7-90a9-4a31-853e-255f855d58d7\") " pod="openshift-service-ca/service-ca-658d7b7c78-p74fm"
Jul 24 15:46:13.484635 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.484530    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf66m\" (UniqueName: \"kubernetes.io/projected/985ed3e7-90a9-4a31-853e-255f855d58d7-kube-api-access-sf66m\") pod \"service-ca-658d7b7c78-p74fm\" (UID: \"985ed3e7-90a9-4a31-853e-255f855d58d7\") " pod="openshift-service-ca/service-ca-658d7b7c78-p74fm"
Jul 24 15:46:13.484635 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.484596    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/985ed3e7-90a9-4a31-853e-255f855d58d7-signing-cabundle\") pod \"service-ca-658d7b7c78-p74fm\" (UID: \"985ed3e7-90a9-4a31-853e-255f855d58d7\") " pod="openshift-service-ca/service-ca-658d7b7c78-p74fm"
Jul 24 15:46:13.485358 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.485333    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/985ed3e7-90a9-4a31-853e-255f855d58d7-signing-cabundle\") pod \"service-ca-658d7b7c78-p74fm\" (UID: \"985ed3e7-90a9-4a31-853e-255f855d58d7\") " pod="openshift-service-ca/service-ca-658d7b7c78-p74fm"
Jul 24 15:46:13.487088 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.487071    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/985ed3e7-90a9-4a31-853e-255f855d58d7-signing-key\") pod \"service-ca-658d7b7c78-p74fm\" (UID: \"985ed3e7-90a9-4a31-853e-255f855d58d7\") " pod="openshift-service-ca/service-ca-658d7b7c78-p74fm"
Jul 24 15:46:13.493038 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.493016    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf66m\" (UniqueName: \"kubernetes.io/projected/985ed3e7-90a9-4a31-853e-255f855d58d7-kube-api-access-sf66m\") pod \"service-ca-658d7b7c78-p74fm\" (UID: \"985ed3e7-90a9-4a31-853e-255f855d58d7\") " pod="openshift-service-ca/service-ca-658d7b7c78-p74fm"
Jul 24 15:46:13.574937 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.574825    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-v8c2j" event={"ID":"568f5f8a-4f51-4104-9d52-f9495186f9ad","Type":"ContainerStarted","Data":"ec9c888e022001b345b704e39b0b899aa47f6d25d6d26f7932d274b90beaf6ab"}
Jul 24 15:46:13.574937 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.574875    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-v8c2j" event={"ID":"568f5f8a-4f51-4104-9d52-f9495186f9ad","Type":"ContainerStarted","Data":"7ff5d4d5957be475cedec23008198886c953d2e115aded0406c0a53a89e17c1b"}
Jul 24 15:46:13.583425 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.583407    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-p74fm"
Jul 24 15:46:13.591801 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.591755    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-v8c2j" podStartSLOduration=1.5597524090000001 podStartE2EDuration="2.591740511s" totalImagesPullingTime="1.031988102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:46:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:46:13.590969265 +0000 UTC m=+52.875296876" watchObservedRunningTime="2026-07-24 15:46:13.591740511 +0000 UTC m=+52.876068119"
Jul 24 15:46:13.703475 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:13.703444    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-p74fm"]
Jul 24 15:46:13.706388 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:46:13.706365    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod985ed3e7_90a9_4a31_853e_255f855d58d7.slice/crio-a49aaedafb9e656300e7ff7fdc7ae6d2afedd0d814dfa748cb7310926d9b8e98 WatchSource:0}: Error finding container a49aaedafb9e656300e7ff7fdc7ae6d2afedd0d814dfa748cb7310926d9b8e98: Status 404 returned error can't find the container with id a49aaedafb9e656300e7ff7fdc7ae6d2afedd0d814dfa748cb7310926d9b8e98
Jul 24 15:46:14.366303 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:14.366270    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4lpwk_ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94/node-ca/0.log"
Jul 24 15:46:14.579145 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:14.579108    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-p74fm" event={"ID":"985ed3e7-90a9-4a31-853e-255f855d58d7","Type":"ContainerStarted","Data":"a49aaedafb9e656300e7ff7fdc7ae6d2afedd0d814dfa748cb7310926d9b8e98"}
Jul 24 15:46:15.166965 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:15.166847    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5jtpt_acb09411-3411-4d23-875c-2b4768a9a968/serve-healthcheck-canary/0.log"
Jul 24 15:46:15.399809 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:15.399768    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:15.399980 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:15.399845    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba905f82-a2c3-4b61-b253-d7a66300d7c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:15.399980 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:15.399899    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:46:15.399980 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:15.399953    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:46:15.399980 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:15.399979    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-rbac-proxy-cm podName:ba905f82-a2c3-4b61-b253-d7a66300d7c2 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:19.399963823 +0000 UTC m=+58.684291408 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z79q4" (UID: "ba905f82-a2c3-4b61-b253-d7a66300d7c2") : configmap "kube-rbac-proxy" not found
Jul 24 15:46:15.400116 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:15.399997    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba905f82-a2c3-4b61-b253-d7a66300d7c2-insights-runtime-extractor-tls podName:ba905f82-a2c3-4b61-b253-d7a66300d7c2 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:19.399986402 +0000 UTC m=+58.684313991 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ba905f82-a2c3-4b61-b253-d7a66300d7c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z79q4" (UID: "ba905f82-a2c3-4b61-b253-d7a66300d7c2") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:46:15.582976 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:15.582937    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-p74fm" event={"ID":"985ed3e7-90a9-4a31-853e-255f855d58d7","Type":"ContainerStarted","Data":"875e8ec962514fa3958b0d59e6c91f20ed89b1a51efc920168d582db3cb642ab"}
Jul 24 15:46:15.598096 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:15.598016    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-p74fm" podStartSLOduration=0.86912046 podStartE2EDuration="2.597998184s" totalImagesPullingTime="1.728877724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:46:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:46:15.597204199 +0000 UTC m=+54.881531807" watchObservedRunningTime="2026-07-24 15:46:15.597998184 +0000 UTC m=+54.882325791"
Jul 24 15:46:17.918496 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:17.918448    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls\") pod \"dns-default-8wnvz\" (UID: \"5746ff45-485c-498a-904e-8c506e3cffc4\") " pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:17.918880 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:17.918605    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 24 15:46:17.918880 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:17.918678    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls podName:5746ff45-485c-498a-904e-8c506e3cffc4 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:33.918661672 +0000 UTC m=+73.202989257 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls") pod "dns-default-8wnvz" (UID: "5746ff45-485c-498a-904e-8c506e3cffc4") : secret "dns-default-metrics-tls" not found
Jul 24 15:46:19.431192 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:19.431147    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:19.431708 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:19.431222    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba905f82-a2c3-4b61-b253-d7a66300d7c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:19.431708 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:19.431298    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 24 15:46:19.431708 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:19.431326    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 24 15:46:19.431708 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:19.431393    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba905f82-a2c3-4b61-b253-d7a66300d7c2-insights-runtime-extractor-tls podName:ba905f82-a2c3-4b61-b253-d7a66300d7c2 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:27.431376127 +0000 UTC m=+66.715703713 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ba905f82-a2c3-4b61-b253-d7a66300d7c2-insights-runtime-extractor-tls") pod "insights-runtime-extractor-z79q4" (UID: "ba905f82-a2c3-4b61-b253-d7a66300d7c2") : secret "insights-runtime-extractor-tls" not found
Jul 24 15:46:19.431708 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:19.431407    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-rbac-proxy-cm podName:ba905f82-a2c3-4b61-b253-d7a66300d7c2 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:27.431401294 +0000 UTC m=+66.715728878 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-rbac-proxy-cm") pod "insights-runtime-extractor-z79q4" (UID: "ba905f82-a2c3-4b61-b253-d7a66300d7c2") : configmap "kube-rbac-proxy" not found
Jul 24 15:46:25.879449 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:25.879401    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret\") pod \"global-pull-secret-syncer-k49df\" (UID: \"79ccf6fc-b5a8-4aff-bcb2-c14898719ac4\") " pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:46:25.882106 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:25.882084    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 24 15:46:25.892904 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:25.892874    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79ccf6fc-b5a8-4aff-bcb2-c14898719ac4-original-pull-secret\") pod \"global-pull-secret-syncer-k49df\" (UID: \"79ccf6fc-b5a8-4aff-bcb2-c14898719ac4\") " pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:46:25.980299 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:25.980261    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs\") pod \"network-metrics-daemon-b86md\" (UID: \"c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e\") " pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:46:25.982742 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:25.982718    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 24 15:46:25.992985 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:25.992953    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e-metrics-certs\") pod \"network-metrics-daemon-b86md\" (UID: \"c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e\") " pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:46:26.035648 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:26.035607    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k49df"
Jul 24 15:46:26.042662 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:26.042630    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pqxfk\""
Jul 24 15:46:26.050531 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:26.050510    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b86md"
Jul 24 15:46:26.081015 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:26.080982    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hflb\" (UniqueName: \"kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb\") pod \"network-check-target-b669x\" (UID: \"7f4a7724-28ad-4bdb-938b-058b8e0bbcf5\") " pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:46:26.083980 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:26.083679    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 24 15:46:26.094402 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:26.094236    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 24 15:46:26.106309 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:26.106286    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hflb\" (UniqueName: \"kubernetes.io/projected/7f4a7724-28ad-4bdb-938b-058b8e0bbcf5-kube-api-access-4hflb\") pod \"network-check-target-b669x\" (UID: \"7f4a7724-28ad-4bdb-938b-058b8e0bbcf5\") " pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:46:26.163112 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:26.163081    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k49df"]
Jul 24 15:46:26.165980 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:46:26.165954    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ccf6fc_b5a8_4aff_bcb2_c14898719ac4.slice/crio-9cac609f03efa5f91a575849c93afba74585bbf7bfe202246fdbdbda45bf0d57 WatchSource:0}: Error finding container 9cac609f03efa5f91a575849c93afba74585bbf7bfe202246fdbdbda45bf0d57: Status 404 returned error can't find the container with id 9cac609f03efa5f91a575849c93afba74585bbf7bfe202246fdbdbda45bf0d57
Jul 24 15:46:26.179788 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:26.179765    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b86md"]
Jul 24 15:46:26.182307 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:46:26.182285    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc85c41d2_09c5_4e4d_aba0_8ccc9bab1f4e.slice/crio-c0c9a5af2d14da17423f5947183c1ea36ce09af3ae610594512a425f8b21db7e WatchSource:0}: Error finding container c0c9a5af2d14da17423f5947183c1ea36ce09af3ae610594512a425f8b21db7e: Status 404 returned error can't find the container with id c0c9a5af2d14da17423f5947183c1ea36ce09af3ae610594512a425f8b21db7e
Jul 24 15:46:26.332124 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:26.332091    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cbp7q\""
Jul 24 15:46:26.340068 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:26.340049    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:46:26.462145 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:26.461946    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b669x"]
Jul 24 15:46:26.464657 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:46:26.464626    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f4a7724_28ad_4bdb_938b_058b8e0bbcf5.slice/crio-775db3e9cd7593e2364eb24db0e42c13cb1b0d876866ce890e1e9bcb48907886 WatchSource:0}: Error finding container 775db3e9cd7593e2364eb24db0e42c13cb1b0d876866ce890e1e9bcb48907886: Status 404 returned error can't find the container with id 775db3e9cd7593e2364eb24db0e42c13cb1b0d876866ce890e1e9bcb48907886
Jul 24 15:46:26.531278 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:26.531252    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lj8js"
Jul 24 15:46:26.613856 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:26.613818    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k49df" event={"ID":"79ccf6fc-b5a8-4aff-bcb2-c14898719ac4","Type":"ContainerStarted","Data":"9cac609f03efa5f91a575849c93afba74585bbf7bfe202246fdbdbda45bf0d57"}
Jul 24 15:46:26.615326 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:26.615290    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b86md" event={"ID":"c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e","Type":"ContainerStarted","Data":"c0c9a5af2d14da17423f5947183c1ea36ce09af3ae610594512a425f8b21db7e"}
Jul 24 15:46:26.616626 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:26.616584    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b669x" event={"ID":"7f4a7724-28ad-4bdb-938b-058b8e0bbcf5","Type":"ContainerStarted","Data":"775db3e9cd7593e2364eb24db0e42c13cb1b0d876866ce890e1e9bcb48907886"}
Jul 24 15:46:27.490229 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:27.490134    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba905f82-a2c3-4b61-b253-d7a66300d7c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:27.490687 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:27.490228    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:27.490739 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:27.490724    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ba905f82-a2c3-4b61-b253-d7a66300d7c2-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:27.493373 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:27.493310    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ba905f82-a2c3-4b61-b253-d7a66300d7c2-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-z79q4\" (UID: \"ba905f82-a2c3-4b61-b253-d7a66300d7c2\") " pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:27.507743 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:27.507715    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vfsm2\""
Jul 24 15:46:27.515779 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:27.515754    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-z79q4"
Jul 24 15:46:27.634495 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:27.634460    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b86md" event={"ID":"c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e","Type":"ContainerStarted","Data":"940929e0edea8c7d0f9c0dd9aa533663ffd775c60a559980e17d7df9e017179f"}
Jul 24 15:46:27.634657 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:27.634526    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b86md" event={"ID":"c85c41d2-09c5-4e4d-aba0-8ccc9bab1f4e","Type":"ContainerStarted","Data":"300f92c12510ecc0d237fd88cb6ea3bca4efaadac84545f6e9aa51118d92b04a"}
Jul 24 15:46:27.651419 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:27.651343    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-b86md" podStartSLOduration=65.655296358 podStartE2EDuration="1m6.65132239s" totalImagesPullingTime="996.026032ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:46:27.650987683 +0000 UTC m=+66.935315291" watchObservedRunningTime="2026-07-24 15:46:27.65132239 +0000 UTC m=+66.935649999"
Jul 24 15:46:27.677339 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:27.677255    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-z79q4"]
Jul 24 15:46:27.681842 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:46:27.681812    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba905f82_a2c3_4b61_b253_d7a66300d7c2.slice/crio-7c9c61f0c7f69eff96be34e1fb1162c2ec9c8feb54921b3f03c3fbd75409257c WatchSource:0}: Error finding container 7c9c61f0c7f69eff96be34e1fb1162c2ec9c8feb54921b3f03c3fbd75409257c: Status 404 returned error can't find the container with id 7c9c61f0c7f69eff96be34e1fb1162c2ec9c8feb54921b3f03c3fbd75409257c
Jul 24 15:46:28.642048 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:28.642012    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z79q4" event={"ID":"ba905f82-a2c3-4b61-b253-d7a66300d7c2","Type":"ContainerStarted","Data":"7c9c61f0c7f69eff96be34e1fb1162c2ec9c8feb54921b3f03c3fbd75409257c"}
Jul 24 15:46:32.260048 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.260001    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx"]
Jul 24 15:46:32.272138 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.272105    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b"]
Jul 24 15:46:32.272312 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.272231    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx"
Jul 24 15:46:32.275437 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.275413    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b"
Jul 24 15:46:32.276596 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.276172    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 24 15:46:32.276596 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.276249    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 24 15:46:32.276596 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.276427    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 24 15:46:32.278873 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.277373    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 24 15:46:32.278873 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.277592    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-vkfpw\""
Jul 24 15:46:32.278873 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.277762    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx"]
Jul 24 15:46:32.278873 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.278386    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 24 15:46:32.279143 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.278952    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b"]
Jul 24 15:46:32.330751 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.330712    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e4a032b-4dad-4e3c-8a0f-3e244100a975-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx\" (UID: \"6e4a032b-4dad-4e3c-8a0f-3e244100a975\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx"
Jul 24 15:46:32.330954 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.330769    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6lbf\" (UniqueName: \"kubernetes.io/projected/6e4a032b-4dad-4e3c-8a0f-3e244100a975-kube-api-access-c6lbf\") pod \"managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx\" (UID: \"6e4a032b-4dad-4e3c-8a0f-3e244100a975\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx"
Jul 24 15:46:32.330954 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.330789    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/860f500b-d4e9-48aa-89d0-0485f7a2b7d2-klusterlet-config\") pod \"klusterlet-addon-workmgr-5b4bc59977-k6g4b\" (UID: \"860f500b-d4e9-48aa-89d0-0485f7a2b7d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b"
Jul 24 15:46:32.330954 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.330844    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96kwq\" (UniqueName: \"kubernetes.io/projected/860f500b-d4e9-48aa-89d0-0485f7a2b7d2-kube-api-access-96kwq\") pod \"klusterlet-addon-workmgr-5b4bc59977-k6g4b\" (UID: \"860f500b-d4e9-48aa-89d0-0485f7a2b7d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b"
Jul 24 15:46:32.331115 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.330977    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/860f500b-d4e9-48aa-89d0-0485f7a2b7d2-tmp\") pod \"klusterlet-addon-workmgr-5b4bc59977-k6g4b\" (UID: \"860f500b-d4e9-48aa-89d0-0485f7a2b7d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b"
Jul 24 15:46:32.353162 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.353127    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"]
Jul 24 15:46:32.356390 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.356370    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.359005 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.358976    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 24 15:46:32.359124 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.359092    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 24 15:46:32.359196 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.359164    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-nsfqx\""
Jul 24 15:46:32.359395 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.359378    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 24 15:46:32.365342 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.365314    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 24 15:46:32.371546 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.371525    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"]
Jul 24 15:46:32.432078 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.432046    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6lbf\" (UniqueName: \"kubernetes.io/projected/6e4a032b-4dad-4e3c-8a0f-3e244100a975-kube-api-access-c6lbf\") pod \"managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx\" (UID: \"6e4a032b-4dad-4e3c-8a0f-3e244100a975\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx"
Jul 24 15:46:32.432255 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.432101    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0bcfb694-899b-4e73-b76e-69e21015bd67-registry-certificates\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.432255 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.432138    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e4a032b-4dad-4e3c-8a0f-3e244100a975-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx\" (UID: \"6e4a032b-4dad-4e3c-8a0f-3e244100a975\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx"
Jul 24 15:46:32.432255 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.432156    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/860f500b-d4e9-48aa-89d0-0485f7a2b7d2-klusterlet-config\") pod \"klusterlet-addon-workmgr-5b4bc59977-k6g4b\" (UID: \"860f500b-d4e9-48aa-89d0-0485f7a2b7d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b"
Jul 24 15:46:32.432255 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.432176    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/860f500b-d4e9-48aa-89d0-0485f7a2b7d2-tmp\") pod \"klusterlet-addon-workmgr-5b4bc59977-k6g4b\" (UID: \"860f500b-d4e9-48aa-89d0-0485f7a2b7d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b"
Jul 24 15:46:32.432255 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.432210    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96kwq\" (UniqueName: \"kubernetes.io/projected/860f500b-d4e9-48aa-89d0-0485f7a2b7d2-kube-api-access-96kwq\") pod \"klusterlet-addon-workmgr-5b4bc59977-k6g4b\" (UID: \"860f500b-d4e9-48aa-89d0-0485f7a2b7d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b"
Jul 24 15:46:32.432255 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.432227    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0bcfb694-899b-4e73-b76e-69e21015bd67-trusted-ca\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.432255 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.432250    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqv5z\" (UniqueName: \"kubernetes.io/projected/0bcfb694-899b-4e73-b76e-69e21015bd67-kube-api-access-kqv5z\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.432611 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.432291    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0bcfb694-899b-4e73-b76e-69e21015bd67-image-registry-private-configuration\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.432611 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.432321    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0bcfb694-899b-4e73-b76e-69e21015bd67-registry-tls\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.432611 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.432349    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0bcfb694-899b-4e73-b76e-69e21015bd67-ca-trust-extracted\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.432611 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.432373    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0bcfb694-899b-4e73-b76e-69e21015bd67-bound-sa-token\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.432611 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.432409    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0bcfb694-899b-4e73-b76e-69e21015bd67-installation-pull-secrets\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.432814 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.432641    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/860f500b-d4e9-48aa-89d0-0485f7a2b7d2-tmp\") pod \"klusterlet-addon-workmgr-5b4bc59977-k6g4b\" (UID: \"860f500b-d4e9-48aa-89d0-0485f7a2b7d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b"
Jul 24 15:46:32.435155 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.435113    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e4a032b-4dad-4e3c-8a0f-3e244100a975-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx\" (UID: \"6e4a032b-4dad-4e3c-8a0f-3e244100a975\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx"
Jul 24 15:46:32.435311 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.435292    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/860f500b-d4e9-48aa-89d0-0485f7a2b7d2-klusterlet-config\") pod \"klusterlet-addon-workmgr-5b4bc59977-k6g4b\" (UID: \"860f500b-d4e9-48aa-89d0-0485f7a2b7d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b"
Jul 24 15:46:32.441765 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.441735    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96kwq\" (UniqueName: \"kubernetes.io/projected/860f500b-d4e9-48aa-89d0-0485f7a2b7d2-kube-api-access-96kwq\") pod \"klusterlet-addon-workmgr-5b4bc59977-k6g4b\" (UID: \"860f500b-d4e9-48aa-89d0-0485f7a2b7d2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b"
Jul 24 15:46:32.441966 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.441944    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6lbf\" (UniqueName: \"kubernetes.io/projected/6e4a032b-4dad-4e3c-8a0f-3e244100a975-kube-api-access-c6lbf\") pod \"managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx\" (UID: \"6e4a032b-4dad-4e3c-8a0f-3e244100a975\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx"
Jul 24 15:46:32.533354 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.533275    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0bcfb694-899b-4e73-b76e-69e21015bd67-image-registry-private-configuration\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.533354 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.533326    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0bcfb694-899b-4e73-b76e-69e21015bd67-registry-tls\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.533569 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.533359    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0bcfb694-899b-4e73-b76e-69e21015bd67-ca-trust-extracted\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.533569 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.533385    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0bcfb694-899b-4e73-b76e-69e21015bd67-bound-sa-token\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.533569 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.533418    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0bcfb694-899b-4e73-b76e-69e21015bd67-installation-pull-secrets\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.533569 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.533466    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0bcfb694-899b-4e73-b76e-69e21015bd67-registry-certificates\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.533569 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.533525    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0bcfb694-899b-4e73-b76e-69e21015bd67-trusted-ca\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.533569 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.533548    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqv5z\" (UniqueName: \"kubernetes.io/projected/0bcfb694-899b-4e73-b76e-69e21015bd67-kube-api-access-kqv5z\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.534194 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.534165    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0bcfb694-899b-4e73-b76e-69e21015bd67-ca-trust-extracted\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.534668 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.534613    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0bcfb694-899b-4e73-b76e-69e21015bd67-registry-certificates\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.534784 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.534719    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0bcfb694-899b-4e73-b76e-69e21015bd67-trusted-ca\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.536390 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.536368    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0bcfb694-899b-4e73-b76e-69e21015bd67-image-registry-private-configuration\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.536690 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.536671    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0bcfb694-899b-4e73-b76e-69e21015bd67-installation-pull-secrets\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.536990 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.536968    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0bcfb694-899b-4e73-b76e-69e21015bd67-registry-tls\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.546603 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.546581    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqv5z\" (UniqueName: \"kubernetes.io/projected/0bcfb694-899b-4e73-b76e-69e21015bd67-kube-api-access-kqv5z\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.546784 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.546757    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0bcfb694-899b-4e73-b76e-69e21015bd67-bound-sa-token\") pod \"image-registry-5f48ddd4fd-n7qdz\" (UID: \"0bcfb694-899b-4e73-b76e-69e21015bd67\") " pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:32.597165 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.597128    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx"
Jul 24 15:46:32.604285 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.604256    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b"
Jul 24 15:46:32.666961 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:32.666905    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:33.945137 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:33.945104    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls\") pod \"dns-default-8wnvz\" (UID: \"5746ff45-485c-498a-904e-8c506e3cffc4\") " pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:33.947887 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:33.947855    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5746ff45-485c-498a-904e-8c506e3cffc4-metrics-tls\") pod \"dns-default-8wnvz\" (UID: \"5746ff45-485c-498a-904e-8c506e3cffc4\") " pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:34.215641 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:34.215603    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vwj8g\""
Jul 24 15:46:34.223757 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:34.223728    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:34.654292 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:34.654160    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx"]
Jul 24 15:46:34.656811 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:46:34.656765    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6e4a032b_4dad_4e3c_8a0f_3e244100a975.slice/crio-3a369a6e53ecd28a247b99d9426699b4b34a76615cdda39b318b10de17cf9175 WatchSource:0}: Error finding container 3a369a6e53ecd28a247b99d9426699b4b34a76615cdda39b318b10de17cf9175: Status 404 returned error can't find the container with id 3a369a6e53ecd28a247b99d9426699b4b34a76615cdda39b318b10de17cf9175
Jul 24 15:46:34.669038 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:34.669002    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b669x" event={"ID":"7f4a7724-28ad-4bdb-938b-058b8e0bbcf5","Type":"ContainerStarted","Data":"56aafc2d57d401f25e77ca20da0121a113d7d5545eb38134b6aba849fc798856"}
Jul 24 15:46:34.669172 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:34.669084    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:46:34.670704 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:34.670668    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z79q4" event={"ID":"ba905f82-a2c3-4b61-b253-d7a66300d7c2","Type":"ContainerStarted","Data":"021046ef7c9033a37157ddaf1a67e7edde3c5b83baff677849b4e1b9bc4911fe"}
Jul 24 15:46:34.671934 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:34.671889    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx" event={"ID":"6e4a032b-4dad-4e3c-8a0f-3e244100a975","Type":"ContainerStarted","Data":"3a369a6e53ecd28a247b99d9426699b4b34a76615cdda39b318b10de17cf9175"}
Jul 24 15:46:34.688872 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:34.688817    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-b669x" podStartSLOduration=65.717279447 podStartE2EDuration="1m13.688799507s" totalImagesPullingTime="7.97152006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:46:34.687793569 +0000 UTC m=+73.972121201" watchObservedRunningTime="2026-07-24 15:46:34.688799507 +0000 UTC m=+73.973127114"
Jul 24 15:46:34.884658 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:34.884628    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b"]
Jul 24 15:46:34.885892 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:34.885793    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8wnvz"]
Jul 24 15:46:34.886889 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:34.886862    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"]
Jul 24 15:46:34.887603 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:46:34.887552    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod860f500b_d4e9_48aa_89d0_0485f7a2b7d2.slice/crio-ab36c66b3d3f9f5ecdbcc5a2257f044a5bb5eed2c12cdf9d471ed06bf4f58c6d WatchSource:0}: Error finding container ab36c66b3d3f9f5ecdbcc5a2257f044a5bb5eed2c12cdf9d471ed06bf4f58c6d: Status 404 returned error can't find the container with id ab36c66b3d3f9f5ecdbcc5a2257f044a5bb5eed2c12cdf9d471ed06bf4f58c6d
Jul 24 15:46:34.887971 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:46:34.887811    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5746ff45_485c_498a_904e_8c506e3cffc4.slice/crio-26120d246b3a644d7b2d917415f291da727b87d7dedfb86b6304b2a2dad7ce86 WatchSource:0}: Error finding container 26120d246b3a644d7b2d917415f291da727b87d7dedfb86b6304b2a2dad7ce86: Status 404 returned error can't find the container with id 26120d246b3a644d7b2d917415f291da727b87d7dedfb86b6304b2a2dad7ce86
Jul 24 15:46:34.889969 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:46:34.889950    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bcfb694_899b_4e73_b76e_69e21015bd67.slice/crio-1d96569e77d5b99e1dab690cf576ba5b0af66ea9a024eb55256de529fd77190f WatchSource:0}: Error finding container 1d96569e77d5b99e1dab690cf576ba5b0af66ea9a024eb55256de529fd77190f: Status 404 returned error can't find the container with id 1d96569e77d5b99e1dab690cf576ba5b0af66ea9a024eb55256de529fd77190f
Jul 24 15:46:35.680735 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:35.680693    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b" event={"ID":"860f500b-d4e9-48aa-89d0-0485f7a2b7d2","Type":"ContainerStarted","Data":"ab36c66b3d3f9f5ecdbcc5a2257f044a5bb5eed2c12cdf9d471ed06bf4f58c6d"}
Jul 24 15:46:35.682366 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:35.682327    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8wnvz" event={"ID":"5746ff45-485c-498a-904e-8c506e3cffc4","Type":"ContainerStarted","Data":"26120d246b3a644d7b2d917415f291da727b87d7dedfb86b6304b2a2dad7ce86"}
Jul 24 15:46:35.685859 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:35.684997    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz" event={"ID":"0bcfb694-899b-4e73-b76e-69e21015bd67","Type":"ContainerStarted","Data":"52ad4b3d9d5c555bc760ae72f1b89fcd6cc2f0464c8b3f586d96bd2d3f245dcf"}
Jul 24 15:46:35.685859 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:35.685049    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz" event={"ID":"0bcfb694-899b-4e73-b76e-69e21015bd67","Type":"ContainerStarted","Data":"1d96569e77d5b99e1dab690cf576ba5b0af66ea9a024eb55256de529fd77190f"}
Jul 24 15:46:35.685859 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:35.685804    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:46:35.690107 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:35.689773    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k49df" event={"ID":"79ccf6fc-b5a8-4aff-bcb2-c14898719ac4","Type":"ContainerStarted","Data":"8a5b5ead736422033cfad2f1e1dac90d08e3a2d3266face56f3487ef7de2f139"}
Jul 24 15:46:35.712142 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:35.711389    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz" podStartSLOduration=3.711367969 podStartE2EDuration="3.711367969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:46:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:46:35.71032231 +0000 UTC m=+74.994649967" watchObservedRunningTime="2026-07-24 15:46:35.711367969 +0000 UTC m=+74.995695577"
Jul 24 15:46:35.725517 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:35.724956    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-k49df" podStartSLOduration=66.451955995 podStartE2EDuration="1m14.724938687s" totalImagesPullingTime="8.272982692s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:46:35.724287014 +0000 UTC m=+75.008614621" watchObservedRunningTime="2026-07-24 15:46:35.724938687 +0000 UTC m=+75.009266288"
Jul 24 15:46:41.715341 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:41.715026    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b" event={"ID":"860f500b-d4e9-48aa-89d0-0485f7a2b7d2","Type":"ContainerStarted","Data":"0fb6b79e932ca1d9e4b539e6f26dad94d15ff52b4ea7c99c9402dcd2a04a0ede"}
Jul 24 15:46:41.715341 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:41.715281    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b"
Jul 24 15:46:41.717439 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:41.716965    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b"
Jul 24 15:46:41.721195 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:41.721144    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx" event={"ID":"6e4a032b-4dad-4e3c-8a0f-3e244100a975","Type":"ContainerStarted","Data":"1588c58ef9b9e801bb64fda0da1a12dcf0cce9a54ba8dd849604960e8102d4d7"}
Jul 24 15:46:41.733604 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:41.733552    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4bc59977-k6g4b" podStartSLOduration=3.05045366 podStartE2EDuration="9.733532257s" totalImagesPullingTime="6.683078597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:46:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:46:41.732584079 +0000 UTC m=+81.016911686" watchObservedRunningTime="2026-07-24 15:46:41.733532257 +0000 UTC m=+81.017859864"
Jul 24 15:46:41.752126 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:41.752066    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-98d9b5d6d-9w4tx" podStartSLOduration=2.82604198 podStartE2EDuration="9.752046631s" totalImagesPullingTime="6.926004651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:46:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:46:41.750646684 +0000 UTC m=+81.034974292" watchObservedRunningTime="2026-07-24 15:46:41.752046631 +0000 UTC m=+81.036374239"
Jul 24 15:46:42.733243 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:42.733199    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z79q4" event={"ID":"ba905f82-a2c3-4b61-b253-d7a66300d7c2","Type":"ContainerStarted","Data":"85d12682bd97ce8c22aafc84360e0def036741e0a0730249e15aad379aa110bb"}
Jul 24 15:46:42.735565 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:42.735106    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8wnvz" event={"ID":"5746ff45-485c-498a-904e-8c506e3cffc4","Type":"ContainerStarted","Data":"8d600aa142881e284aa3faa81da86368e2bdf7cb166885493f5dbd49353c1a62"}
Jul 24 15:46:42.735565 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:42.735155    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8wnvz" event={"ID":"5746ff45-485c-498a-904e-8c506e3cffc4","Type":"ContainerStarted","Data":"14781cba73e5ae00e35767b7e34549f5ba16a9f9f88f7d73aea5711d7cd9e597"}
Jul 24 15:46:42.735565 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:42.735474    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:42.753608 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:42.753558    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8wnvz" podStartSLOduration=34.070581275 podStartE2EDuration="40.753539209s" totalImagesPullingTime="6.682957934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:46:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:46:42.752444994 +0000 UTC m=+82.036772602" watchObservedRunningTime="2026-07-24 15:46:42.753539209 +0000 UTC m=+82.037866820"
Jul 24 15:46:45.747213 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:45.747174    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-z79q4" event={"ID":"ba905f82-a2c3-4b61-b253-d7a66300d7c2","Type":"ContainerStarted","Data":"ff751bff204e4b2b023b97a6d0b5477c5f71c083ae665795c8eb5deec5c60aa6"}
Jul 24 15:46:45.766642 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:45.766592    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-z79q4" podStartSLOduration=17.503936241 podStartE2EDuration="34.76657873s" totalImagesPullingTime="17.262642489s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 15:46:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:46:45.765385956 +0000 UTC m=+85.049713577" watchObservedRunningTime="2026-07-24 15:46:45.76657873 +0000 UTC m=+85.050906337"
Jul 24 15:46:51.138135 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.138090    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bcp9q"]
Jul 24 15:46:51.141578 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.141550    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.145424 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.145400    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 24 15:46:51.145788 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.145657    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 24 15:46:51.146251 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.146224    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 24 15:46:51.146426 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.146233    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 24 15:46:51.146528 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.146274    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 24 15:46:51.146600 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.146335    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 24 15:46:51.146600 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.146409    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7fcwn\""
Jul 24 15:46:51.147015 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.146993    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"]
Jul 24 15:46:51.149494 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.149475    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.152584 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.152565    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 24 15:46:51.152692 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.152675    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 24 15:46:51.153203 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.152780    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-d7dfq\""
Jul 24 15:46:51.153203 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.152899    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 24 15:46:51.164195 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.164173    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"]
Jul 24 15:46:51.279719 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.279681    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f086dcc-d843-4f13-b0e2-b42c11329867-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.279960 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.279728    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-node-exporter-tls\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.279960 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.279763    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-node-exporter-textfile\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.279960 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.279852    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-sys\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.279960 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.279892    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncmmf\" (UniqueName: \"kubernetes.io/projected/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-kube-api-access-ncmmf\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.279960 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.279934    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-metrics-client-ca\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.280211 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.279981    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f086dcc-d843-4f13-b0e2-b42c11329867-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.280211 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.280018    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-root\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.280211 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.280041    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.280211 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.280087    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3f086dcc-d843-4f13-b0e2-b42c11329867-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.280211 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.280114    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-node-exporter-accelerators-collector-config\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.280211 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.280140    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv689\" (UniqueName: \"kubernetes.io/projected/3f086dcc-d843-4f13-b0e2-b42c11329867-kube-api-access-wv689\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.280211 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.280166    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f086dcc-d843-4f13-b0e2-b42c11329867-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.280532 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.280216    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-node-exporter-wtmp\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.280532 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.280280    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3f086dcc-d843-4f13-b0e2-b42c11329867-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.380826 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.380793    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3f086dcc-d843-4f13-b0e2-b42c11329867-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.381023 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.380835    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f086dcc-d843-4f13-b0e2-b42c11329867-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.381023 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.380852    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-node-exporter-tls\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.381023 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.380874    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-node-exporter-textfile\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.381023 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.380906    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-sys\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.381023 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.380953    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncmmf\" (UniqueName: \"kubernetes.io/projected/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-kube-api-access-ncmmf\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.381023 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.380980    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-metrics-client-ca\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.381023 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:51.380994    2572 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 24 15:46:51.381023 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.381018    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f086dcc-d843-4f13-b0e2-b42c11329867-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.381424 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:51.381061    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f086dcc-d843-4f13-b0e2-b42c11329867-kube-state-metrics-tls podName:3f086dcc-d843-4f13-b0e2-b42c11329867 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:51.881037535 +0000 UTC m=+91.165365137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3f086dcc-d843-4f13-b0e2-b42c11329867-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-pkj5s" (UID: "3f086dcc-d843-4f13-b0e2-b42c11329867") : secret "kube-state-metrics-tls" not found
Jul 24 15:46:51.381424 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.381089    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-root\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.381424 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.381115    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.381424 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.381155    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3f086dcc-d843-4f13-b0e2-b42c11329867-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.381424 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.381178    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-node-exporter-accelerators-collector-config\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.381424 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.381206    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv689\" (UniqueName: \"kubernetes.io/projected/3f086dcc-d843-4f13-b0e2-b42c11329867-kube-api-access-wv689\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.381424 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.381234    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f086dcc-d843-4f13-b0e2-b42c11329867-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.381424 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.381260    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-node-exporter-wtmp\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.381812 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.381422    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-node-exporter-wtmp\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.381812 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.381659    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3f086dcc-d843-4f13-b0e2-b42c11329867-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.381812 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.381729    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-sys\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.381986 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.381891    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-root\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.382035 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:51.382023    2572 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 24 15:46:51.382089 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:46:51.382075    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-node-exporter-tls podName:3e1253f5-c2bc-4076-b9d9-c2ac082bfa45 nodeName:}" failed. No retries permitted until 2026-07-24 15:46:51.882056311 +0000 UTC m=+91.166383900 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-node-exporter-tls") pod "node-exporter-bcp9q" (UID: "3e1253f5-c2bc-4076-b9d9-c2ac082bfa45") : secret "node-exporter-tls" not found
Jul 24 15:46:51.383227 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.382512    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-node-exporter-accelerators-collector-config\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.383227 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.382539    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-node-exporter-textfile\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.383227 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.382870    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3f086dcc-d843-4f13-b0e2-b42c11329867-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.383227 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.383099    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-metrics-client-ca\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.383227 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.383189    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3f086dcc-d843-4f13-b0e2-b42c11329867-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.385268 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.385236    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.385408 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.385310    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3f086dcc-d843-4f13-b0e2-b42c11329867-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.390521 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.390449    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncmmf\" (UniqueName: \"kubernetes.io/projected/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-kube-api-access-ncmmf\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.391040 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.391020    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv689\" (UniqueName: \"kubernetes.io/projected/3f086dcc-d843-4f13-b0e2-b42c11329867-kube-api-access-wv689\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.886315 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.886270    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f086dcc-d843-4f13-b0e2-b42c11329867-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:51.886315 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.886315    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-node-exporter-tls\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.888831 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.888802    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3e1253f5-c2bc-4076-b9d9-c2ac082bfa45-node-exporter-tls\") pod \"node-exporter-bcp9q\" (UID: \"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45\") " pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:51.889016 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:51.888996    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f086dcc-d843-4f13-b0e2-b42c11329867-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-pkj5s\" (UID: \"3f086dcc-d843-4f13-b0e2-b42c11329867\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:52.053743 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:52.053707    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bcp9q"
Jul 24 15:46:52.062966 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:52.062941    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"
Jul 24 15:46:52.064550 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:46:52.064519    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e1253f5_c2bc_4076_b9d9_c2ac082bfa45.slice/crio-dfb041ea0a2bcc3f6e3f580dd99636e066fb9d5decde5ef569246ac131013c60 WatchSource:0}: Error finding container dfb041ea0a2bcc3f6e3f580dd99636e066fb9d5decde5ef569246ac131013c60: Status 404 returned error can't find the container with id dfb041ea0a2bcc3f6e3f580dd99636e066fb9d5decde5ef569246ac131013c60
Jul 24 15:46:52.201429 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:52.201398    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s"]
Jul 24 15:46:52.204910 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:46:52.204863    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f086dcc_d843_4f13_b0e2_b42c11329867.slice/crio-ab0bb158905cd4a7c4d8190ad0a49ff290a520b0d51526ecccc975875d72c16f WatchSource:0}: Error finding container ab0bb158905cd4a7c4d8190ad0a49ff290a520b0d51526ecccc975875d72c16f: Status 404 returned error can't find the container with id ab0bb158905cd4a7c4d8190ad0a49ff290a520b0d51526ecccc975875d72c16f
Jul 24 15:46:52.673376 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:52.673338    2572 patch_prober.go:28] interesting pod/image-registry-5f48ddd4fd-n7qdz container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 24 15:46:52.673567 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:52.673402    2572 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz" podUID="0bcfb694-899b-4e73-b76e-69e21015bd67" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 15:46:52.741785 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:52.741754    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8wnvz"
Jul 24 15:46:52.773200 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:52.773155    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s" event={"ID":"3f086dcc-d843-4f13-b0e2-b42c11329867","Type":"ContainerStarted","Data":"ab0bb158905cd4a7c4d8190ad0a49ff290a520b0d51526ecccc975875d72c16f"}
Jul 24 15:46:52.774788 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:52.774754    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bcp9q" event={"ID":"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45","Type":"ContainerStarted","Data":"dfb041ea0a2bcc3f6e3f580dd99636e066fb9d5decde5ef569246ac131013c60"}
Jul 24 15:46:53.782717 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:53.782623    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s" event={"ID":"3f086dcc-d843-4f13-b0e2-b42c11329867","Type":"ContainerStarted","Data":"b3646c685bf455f109095b894f9c0ebf7e60ed5f043f85f56494362f00d06b96"}
Jul 24 15:46:53.782717 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:53.782695    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s" event={"ID":"3f086dcc-d843-4f13-b0e2-b42c11329867","Type":"ContainerStarted","Data":"4465d071ba44e590f21fce40a4d07ac426f29c5a490259b903e0d0a96e656d26"}
Jul 24 15:46:53.782717 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:53.782718    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s" event={"ID":"3f086dcc-d843-4f13-b0e2-b42c11329867","Type":"ContainerStarted","Data":"af8d52965a03854f8088352450b88672cc27ea1194e0ab432107ab6f030a1234"}
Jul 24 15:46:53.784598 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:53.784557    2572 generic.go:356] "Generic (PLEG): container finished" podID="3e1253f5-c2bc-4076-b9d9-c2ac082bfa45" containerID="06dad7ab1da9f5395e5832b5bc06fe9b12d15b28530d1b3e0481acea173b95bb" exitCode=0
Jul 24 15:46:53.784744 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:53.784623    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bcp9q" event={"ID":"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45","Type":"ContainerDied","Data":"06dad7ab1da9f5395e5832b5bc06fe9b12d15b28530d1b3e0481acea173b95bb"}
Jul 24 15:46:53.804998 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:53.804947    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-pkj5s" podStartSLOduration=1.46424899 podStartE2EDuration="2.804911164s" totalImagesPullingTime="1.340662174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:46:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:46:53.803971509 +0000 UTC m=+93.088299119" watchObservedRunningTime="2026-07-24 15:46:53.804911164 +0000 UTC m=+93.089238770"
Jul 24 15:46:54.790458 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:54.790421    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bcp9q" event={"ID":"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45","Type":"ContainerStarted","Data":"138a6b307257008c7a8048b0d9d1fb650f288fdc286445c2023ae038b9e385c4"}
Jul 24 15:46:54.790839 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:54.790480    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bcp9q" event={"ID":"3e1253f5-c2bc-4076-b9d9-c2ac082bfa45","Type":"ContainerStarted","Data":"27ae17e7c9ebd45482babc73284ff67508dff9f551d5fd6a143ed61aaa99f3e0"}
Jul 24 15:46:54.830589 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:54.830535    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bcp9q" podStartSLOduration=2.736249089 podStartE2EDuration="3.830519362s" totalImagesPullingTime="1.018259911s" totalInitContainerRuntime="76.010362ms" isStatefulPod=true podCreationTimestamp="2026-07-24 15:46:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:46:54.829275117 +0000 UTC m=+94.113602726" watchObservedRunningTime="2026-07-24 15:46:54.830519362 +0000 UTC m=+94.114846968"
Jul 24 15:46:57.702668 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:46:57.702639    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5f48ddd4fd-n7qdz"
Jul 24 15:47:03.184663 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.184625    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75bfd965d8-gxk46"]
Jul 24 15:47:03.189952 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.189932    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.192732 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.192707    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 24 15:47:03.192732 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.192720    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 24 15:47:03.193677 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.193652    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4nfqn\""
Jul 24 15:47:03.193837 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.193819    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 24 15:47:03.193936 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.193835    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 24 15:47:03.193936 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.193819    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 24 15:47:03.194052 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.193946    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 24 15:47:03.194052 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.193956    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 24 15:47:03.198830 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.198809    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75bfd965d8-gxk46"]
Jul 24 15:47:03.277711 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.277680    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6455b82-7d22-4102-b7ff-2449e0282359-oauth-serving-cert\") pod \"console-75bfd965d8-gxk46\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") " pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.277711 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.277717    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6455b82-7d22-4102-b7ff-2449e0282359-console-serving-cert\") pod \"console-75bfd965d8-gxk46\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") " pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.277929 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.277735    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wfrx\" (UniqueName: \"kubernetes.io/projected/f6455b82-7d22-4102-b7ff-2449e0282359-kube-api-access-7wfrx\") pod \"console-75bfd965d8-gxk46\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") " pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.277929 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.277756    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6455b82-7d22-4102-b7ff-2449e0282359-console-oauth-config\") pod \"console-75bfd965d8-gxk46\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") " pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.277929 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.277798    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6455b82-7d22-4102-b7ff-2449e0282359-console-config\") pod \"console-75bfd965d8-gxk46\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") " pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.277929 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.277876    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6455b82-7d22-4102-b7ff-2449e0282359-service-ca\") pod \"console-75bfd965d8-gxk46\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") " pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.379211 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.379177    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6455b82-7d22-4102-b7ff-2449e0282359-oauth-serving-cert\") pod \"console-75bfd965d8-gxk46\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") " pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.379211 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.379216    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6455b82-7d22-4102-b7ff-2449e0282359-console-serving-cert\") pod \"console-75bfd965d8-gxk46\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") " pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.379397 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.379234    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wfrx\" (UniqueName: \"kubernetes.io/projected/f6455b82-7d22-4102-b7ff-2449e0282359-kube-api-access-7wfrx\") pod \"console-75bfd965d8-gxk46\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") " pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.379397 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.379254    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6455b82-7d22-4102-b7ff-2449e0282359-console-oauth-config\") pod \"console-75bfd965d8-gxk46\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") " pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.379473 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.379386    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6455b82-7d22-4102-b7ff-2449e0282359-console-config\") pod \"console-75bfd965d8-gxk46\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") " pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.379473 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.379430    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6455b82-7d22-4102-b7ff-2449e0282359-service-ca\") pod \"console-75bfd965d8-gxk46\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") " pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.379978 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.379951    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6455b82-7d22-4102-b7ff-2449e0282359-oauth-serving-cert\") pod \"console-75bfd965d8-gxk46\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") " pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.380245 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.380082    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6455b82-7d22-4102-b7ff-2449e0282359-service-ca\") pod \"console-75bfd965d8-gxk46\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") " pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.380299 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.380252    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6455b82-7d22-4102-b7ff-2449e0282359-console-config\") pod \"console-75bfd965d8-gxk46\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") " pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.381881 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.381860    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6455b82-7d22-4102-b7ff-2449e0282359-console-serving-cert\") pod \"console-75bfd965d8-gxk46\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") " pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.382006 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.381985    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6455b82-7d22-4102-b7ff-2449e0282359-console-oauth-config\") pod \"console-75bfd965d8-gxk46\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") " pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.388305 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.388285    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wfrx\" (UniqueName: \"kubernetes.io/projected/f6455b82-7d22-4102-b7ff-2449e0282359-kube-api-access-7wfrx\") pod \"console-75bfd965d8-gxk46\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") " pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.500852 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.500764    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:03.637040 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.636983    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75bfd965d8-gxk46"]
Jul 24 15:47:03.639655 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:47:03.639626    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6455b82_7d22_4102_b7ff_2449e0282359.slice/crio-90060c2d41b595aa3c58535f9cecca098560443c652a8ac8fbcc9ff5ab5065ac WatchSource:0}: Error finding container 90060c2d41b595aa3c58535f9cecca098560443c652a8ac8fbcc9ff5ab5065ac: Status 404 returned error can't find the container with id 90060c2d41b595aa3c58535f9cecca098560443c652a8ac8fbcc9ff5ab5065ac
Jul 24 15:47:03.819767 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:03.819675    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75bfd965d8-gxk46" event={"ID":"f6455b82-7d22-4102-b7ff-2449e0282359","Type":"ContainerStarted","Data":"90060c2d41b595aa3c58535f9cecca098560443c652a8ac8fbcc9ff5ab5065ac"}
Jul 24 15:47:05.692792 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:05.692754    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-b669x"
Jul 24 15:47:07.837733 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:07.837699    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75bfd965d8-gxk46" event={"ID":"f6455b82-7d22-4102-b7ff-2449e0282359","Type":"ContainerStarted","Data":"d4d9a58b988c6869fadd4e1b61c7a6624b6a74116aa9a00e28edf99e0ba77282"}
Jul 24 15:47:07.857050 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:07.856999    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75bfd965d8-gxk46" podStartSLOduration=1.3756842329999999 podStartE2EDuration="4.856981689s" totalImagesPullingTime="3.481297456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:47:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:47:07.856208282 +0000 UTC m=+107.140535908" watchObservedRunningTime="2026-07-24 15:47:07.856981689 +0000 UTC m=+107.141309296"
Jul 24 15:47:13.500934 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:13.500889    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:13.501434 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:13.500985    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:13.506225 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:13.506194    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:13.862269 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:13.862182    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:19.568305 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:19.568273    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-pkj5s_3f086dcc-d843-4f13-b0e2-b42c11329867/kube-state-metrics/0.log"
Jul 24 15:47:19.767265 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:19.767236    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-pkj5s_3f086dcc-d843-4f13-b0e2-b42c11329867/kube-rbac-proxy-main/0.log"
Jul 24 15:47:19.967412 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:19.967379    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-pkj5s_3f086dcc-d843-4f13-b0e2-b42c11329867/kube-rbac-proxy-self/0.log"
Jul 24 15:47:20.568234 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:20.568204    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bcp9q_3e1253f5-c2bc-4076-b9d9-c2ac082bfa45/init-textfile/0.log"
Jul 24 15:47:20.769301 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:20.769270    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bcp9q_3e1253f5-c2bc-4076-b9d9-c2ac082bfa45/node-exporter/0.log"
Jul 24 15:47:20.967786 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:20.967747    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bcp9q_3e1253f5-c2bc-4076-b9d9-c2ac082bfa45/kube-rbac-proxy/0.log"
Jul 24 15:47:21.369231 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:21.369145    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75bfd965d8-gxk46"]
Jul 24 15:47:26.968426 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:26.968385    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-75bfd965d8-gxk46_f6455b82-7d22-4102-b7ff-2449e0282359/console/0.log"
Jul 24 15:47:27.567521 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:27.567476    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5jtpt_acb09411-3411-4d23-875c-2b4768a9a968/serve-healthcheck-canary/0.log"
Jul 24 15:47:46.389156 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.389073    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75bfd965d8-gxk46" podUID="f6455b82-7d22-4102-b7ff-2449e0282359" containerName="console" containerID="cri-o://d4d9a58b988c6869fadd4e1b61c7a6624b6a74116aa9a00e28edf99e0ba77282" gracePeriod=15
Jul 24 15:47:46.624470 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.624447    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:46.721991 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.721950    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6455b82-7d22-4102-b7ff-2449e0282359-console-serving-cert\") pod \"f6455b82-7d22-4102-b7ff-2449e0282359\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") "
Jul 24 15:47:46.721991 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.721998    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6455b82-7d22-4102-b7ff-2449e0282359-oauth-serving-cert\") pod \"f6455b82-7d22-4102-b7ff-2449e0282359\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") "
Jul 24 15:47:46.722232 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.722042    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6455b82-7d22-4102-b7ff-2449e0282359-console-oauth-config\") pod \"f6455b82-7d22-4102-b7ff-2449e0282359\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") "
Jul 24 15:47:46.722232 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.722068    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wfrx\" (UniqueName: \"kubernetes.io/projected/f6455b82-7d22-4102-b7ff-2449e0282359-kube-api-access-7wfrx\") pod \"f6455b82-7d22-4102-b7ff-2449e0282359\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") "
Jul 24 15:47:46.722232 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.722135    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6455b82-7d22-4102-b7ff-2449e0282359-console-config\") pod \"f6455b82-7d22-4102-b7ff-2449e0282359\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") "
Jul 24 15:47:46.722232 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.722168    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6455b82-7d22-4102-b7ff-2449e0282359-service-ca\") pod \"f6455b82-7d22-4102-b7ff-2449e0282359\" (UID: \"f6455b82-7d22-4102-b7ff-2449e0282359\") "
Jul 24 15:47:46.722553 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.722522    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6455b82-7d22-4102-b7ff-2449e0282359-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f6455b82-7d22-4102-b7ff-2449e0282359" (UID: "f6455b82-7d22-4102-b7ff-2449e0282359"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:47:46.722643 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.722573    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6455b82-7d22-4102-b7ff-2449e0282359-console-config" (OuterVolumeSpecName: "console-config") pod "f6455b82-7d22-4102-b7ff-2449e0282359" (UID: "f6455b82-7d22-4102-b7ff-2449e0282359"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:47:46.722700 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.722634    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6455b82-7d22-4102-b7ff-2449e0282359-service-ca" (OuterVolumeSpecName: "service-ca") pod "f6455b82-7d22-4102-b7ff-2449e0282359" (UID: "f6455b82-7d22-4102-b7ff-2449e0282359"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:47:46.724399 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.724368    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6455b82-7d22-4102-b7ff-2449e0282359-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f6455b82-7d22-4102-b7ff-2449e0282359" (UID: "f6455b82-7d22-4102-b7ff-2449e0282359"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:47:46.724500 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.724480    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6455b82-7d22-4102-b7ff-2449e0282359-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f6455b82-7d22-4102-b7ff-2449e0282359" (UID: "f6455b82-7d22-4102-b7ff-2449e0282359"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:47:46.724598 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.724569    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6455b82-7d22-4102-b7ff-2449e0282359-kube-api-access-7wfrx" (OuterVolumeSpecName: "kube-api-access-7wfrx") pod "f6455b82-7d22-4102-b7ff-2449e0282359" (UID: "f6455b82-7d22-4102-b7ff-2449e0282359"). InnerVolumeSpecName "kube-api-access-7wfrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:47:46.823639 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.823600    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f6455b82-7d22-4102-b7ff-2449e0282359-console-config\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 15:47:46.823639 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.823631    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f6455b82-7d22-4102-b7ff-2449e0282359-service-ca\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 15:47:46.823639 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.823640    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f6455b82-7d22-4102-b7ff-2449e0282359-console-serving-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 15:47:46.823639 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.823650    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f6455b82-7d22-4102-b7ff-2449e0282359-oauth-serving-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 15:47:46.823899 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.823658    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f6455b82-7d22-4102-b7ff-2449e0282359-console-oauth-config\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 15:47:46.823899 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.823666    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wfrx\" (UniqueName: \"kubernetes.io/projected/f6455b82-7d22-4102-b7ff-2449e0282359-kube-api-access-7wfrx\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 15:47:46.979719 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.979622    2572 generic.go:356] "Generic (PLEG): container finished" podID="f6455b82-7d22-4102-b7ff-2449e0282359" containerID="d4d9a58b988c6869fadd4e1b61c7a6624b6a74116aa9a00e28edf99e0ba77282" exitCode=0
Jul 24 15:47:46.979719 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.979673    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75bfd965d8-gxk46"
Jul 24 15:47:46.979906 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.979719    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75bfd965d8-gxk46" event={"ID":"f6455b82-7d22-4102-b7ff-2449e0282359","Type":"ContainerDied","Data":"d4d9a58b988c6869fadd4e1b61c7a6624b6a74116aa9a00e28edf99e0ba77282"}
Jul 24 15:47:46.979906 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.979780    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75bfd965d8-gxk46" event={"ID":"f6455b82-7d22-4102-b7ff-2449e0282359","Type":"ContainerDied","Data":"90060c2d41b595aa3c58535f9cecca098560443c652a8ac8fbcc9ff5ab5065ac"}
Jul 24 15:47:46.979906 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.979804    2572 scope.go:117] "RemoveContainer" containerID="d4d9a58b988c6869fadd4e1b61c7a6624b6a74116aa9a00e28edf99e0ba77282"
Jul 24 15:47:46.989096 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.989077    2572 scope.go:117] "RemoveContainer" containerID="d4d9a58b988c6869fadd4e1b61c7a6624b6a74116aa9a00e28edf99e0ba77282"
Jul 24 15:47:46.989372 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:47:46.989349    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4d9a58b988c6869fadd4e1b61c7a6624b6a74116aa9a00e28edf99e0ba77282\": container with ID starting with d4d9a58b988c6869fadd4e1b61c7a6624b6a74116aa9a00e28edf99e0ba77282 not found: ID does not exist" containerID="d4d9a58b988c6869fadd4e1b61c7a6624b6a74116aa9a00e28edf99e0ba77282"
Jul 24 15:47:46.989423 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:46.989381    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4d9a58b988c6869fadd4e1b61c7a6624b6a74116aa9a00e28edf99e0ba77282"} err="failed to get container status \"d4d9a58b988c6869fadd4e1b61c7a6624b6a74116aa9a00e28edf99e0ba77282\": rpc error: code = NotFound desc = could not find container \"d4d9a58b988c6869fadd4e1b61c7a6624b6a74116aa9a00e28edf99e0ba77282\": container with ID starting with d4d9a58b988c6869fadd4e1b61c7a6624b6a74116aa9a00e28edf99e0ba77282 not found: ID does not exist"
Jul 24 15:47:47.001353 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:47.001325    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75bfd965d8-gxk46"]
Jul 24 15:47:47.003464 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:47.003440    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75bfd965d8-gxk46"]
Jul 24 15:47:47.222037 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:47:47.222005    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6455b82-7d22-4102-b7ff-2449e0282359" path="/var/lib/kubelet/pods/f6455b82-7d22-4102-b7ff-2449e0282359/volumes"
Jul 24 15:48:19.375464 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.375380    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7fbff8c467-8h4kb"]
Jul 24 15:48:19.375890 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.375650    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6455b82-7d22-4102-b7ff-2449e0282359" containerName="console"
Jul 24 15:48:19.375890 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.375662    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6455b82-7d22-4102-b7ff-2449e0282359" containerName="console"
Jul 24 15:48:19.375890 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.375711    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6455b82-7d22-4102-b7ff-2449e0282359" containerName="console"
Jul 24 15:48:19.378570 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.378551    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.381577 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.381553    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 24 15:48:19.381687 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.381562    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4nfqn\""
Jul 24 15:48:19.382410 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.382389    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 24 15:48:19.382505 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.382426    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 24 15:48:19.382505 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.382396    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 24 15:48:19.382793 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.382777    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 24 15:48:19.383552 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.383537    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 24 15:48:19.383633 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.383583    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 24 15:48:19.386700 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.386680    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 24 15:48:19.391685 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.391663    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fbff8c467-8h4kb"]
Jul 24 15:48:19.469552 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.469512    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9hwz\" (UniqueName: \"kubernetes.io/projected/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-kube-api-access-q9hwz\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.469552 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.469557    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-console-config\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.469790 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.469588    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-service-ca\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.469790 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.469616    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-console-serving-cert\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.469790 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.469641    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-console-oauth-config\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.469790 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.469667    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-oauth-serving-cert\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.469790 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.469693    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-trusted-ca-bundle\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.571004 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.570962    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9hwz\" (UniqueName: \"kubernetes.io/projected/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-kube-api-access-q9hwz\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.571004 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.571007    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-console-config\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.571243 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.571140    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-service-ca\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.571243 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.571171    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-console-serving-cert\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.571243 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.571197    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-console-oauth-config\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.571243 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.571235    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-oauth-serving-cert\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.571443 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.571271    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-trusted-ca-bundle\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.571824 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.571793    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-console-config\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.572166 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.571953    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-oauth-serving-cert\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.572166 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.572157    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-trusted-ca-bundle\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.572295 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.572164    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-service-ca\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.573907 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.573886    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-console-oauth-config\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.574044 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.573892    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-console-serving-cert\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.579196 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.579169    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9hwz\" (UniqueName: \"kubernetes.io/projected/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-kube-api-access-q9hwz\") pod \"console-7fbff8c467-8h4kb\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") " pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.687331 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.687294    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:19.819009 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:19.818983    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fbff8c467-8h4kb"]
Jul 24 15:48:19.821427 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:48:19.821386    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1589bd8c_b8bd_42a5_b12a_4e6d1b9ab787.slice/crio-3ebe23139cf6f44b679dc1d2600d3452e360f18616284798558c0ee0f890506a WatchSource:0}: Error finding container 3ebe23139cf6f44b679dc1d2600d3452e360f18616284798558c0ee0f890506a: Status 404 returned error can't find the container with id 3ebe23139cf6f44b679dc1d2600d3452e360f18616284798558c0ee0f890506a
Jul 24 15:48:20.093998 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:20.093887    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fbff8c467-8h4kb" event={"ID":"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787","Type":"ContainerStarted","Data":"6e0447170051172a6683db51500a92b3ea33a3583e72d3938650b55396e56fc6"}
Jul 24 15:48:20.093998 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:20.093962    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fbff8c467-8h4kb" event={"ID":"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787","Type":"ContainerStarted","Data":"3ebe23139cf6f44b679dc1d2600d3452e360f18616284798558c0ee0f890506a"}
Jul 24 15:48:20.111702 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:20.111655    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7fbff8c467-8h4kb" podStartSLOduration=1.111640222 podStartE2EDuration="1.111640222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:48:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:48:20.110671553 +0000 UTC m=+179.394999159" watchObservedRunningTime="2026-07-24 15:48:20.111640222 +0000 UTC m=+179.395967828"
Jul 24 15:48:29.687750 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:29.687713    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:29.688270 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:29.687783    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:29.693509 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:29.693483    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:48:30.131021 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:48:30.130943    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:49:41.960046 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:49:41.959953    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fbff8c467-8h4kb"]
Jul 24 15:50:06.979101 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:06.979056    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7fbff8c467-8h4kb" podUID="1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787" containerName="console" containerID="cri-o://6e0447170051172a6683db51500a92b3ea33a3583e72d3938650b55396e56fc6" gracePeriod=15
Jul 24 15:50:07.218745 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.218722    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:50:07.377626 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.377515    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-console-config\") pod \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") "
Jul 24 15:50:07.377626 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.377580    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-trusted-ca-bundle\") pod \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") "
Jul 24 15:50:07.377626 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.377619    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-console-serving-cert\") pod \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") "
Jul 24 15:50:07.377958 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.377716    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-service-ca\") pod \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") "
Jul 24 15:50:07.377958 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.377771    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-console-oauth-config\") pod \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") "
Jul 24 15:50:07.377958 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.377802    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-oauth-serving-cert\") pod \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") "
Jul 24 15:50:07.377958 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.377860    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9hwz\" (UniqueName: \"kubernetes.io/projected/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-kube-api-access-q9hwz\") pod \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\" (UID: \"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787\") "
Jul 24 15:50:07.378167 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.378081    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-console-config" (OuterVolumeSpecName: "console-config") pod "1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787" (UID: "1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:50:07.378167 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.378143    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-service-ca" (OuterVolumeSpecName: "service-ca") pod "1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787" (UID: "1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:50:07.378249 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.378072    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787" (UID: "1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:50:07.378293 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.378244    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787" (UID: "1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 15:50:07.379963 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.379939    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787" (UID: "1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:50:07.380084 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.380058    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787" (UID: "1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:50:07.380181 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.380110    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-kube-api-access-q9hwz" (OuterVolumeSpecName: "kube-api-access-q9hwz") pod "1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787" (UID: "1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787"). InnerVolumeSpecName "kube-api-access-q9hwz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:50:07.458049 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.458015    2572 generic.go:356] "Generic (PLEG): container finished" podID="1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787" containerID="6e0447170051172a6683db51500a92b3ea33a3583e72d3938650b55396e56fc6" exitCode=0
Jul 24 15:50:07.458220 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.458059    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fbff8c467-8h4kb"
Jul 24 15:50:07.458220 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.458075    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fbff8c467-8h4kb" event={"ID":"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787","Type":"ContainerDied","Data":"6e0447170051172a6683db51500a92b3ea33a3583e72d3938650b55396e56fc6"}
Jul 24 15:50:07.458220 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.458123    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fbff8c467-8h4kb" event={"ID":"1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787","Type":"ContainerDied","Data":"3ebe23139cf6f44b679dc1d2600d3452e360f18616284798558c0ee0f890506a"}
Jul 24 15:50:07.458220 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.458147    2572 scope.go:117] "RemoveContainer" containerID="6e0447170051172a6683db51500a92b3ea33a3583e72d3938650b55396e56fc6"
Jul 24 15:50:07.467080 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.467059    2572 scope.go:117] "RemoveContainer" containerID="6e0447170051172a6683db51500a92b3ea33a3583e72d3938650b55396e56fc6"
Jul 24 15:50:07.467416 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:50:07.467394    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e0447170051172a6683db51500a92b3ea33a3583e72d3938650b55396e56fc6\": container with ID starting with 6e0447170051172a6683db51500a92b3ea33a3583e72d3938650b55396e56fc6 not found: ID does not exist" containerID="6e0447170051172a6683db51500a92b3ea33a3583e72d3938650b55396e56fc6"
Jul 24 15:50:07.467467 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.467427    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e0447170051172a6683db51500a92b3ea33a3583e72d3938650b55396e56fc6"} err="failed to get container status \"6e0447170051172a6683db51500a92b3ea33a3583e72d3938650b55396e56fc6\": rpc error: code = NotFound desc = could not find container \"6e0447170051172a6683db51500a92b3ea33a3583e72d3938650b55396e56fc6\": container with ID starting with 6e0447170051172a6683db51500a92b3ea33a3583e72d3938650b55396e56fc6 not found: ID does not exist"
Jul 24 15:50:07.478799 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.478770    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9hwz\" (UniqueName: \"kubernetes.io/projected/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-kube-api-access-q9hwz\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 15:50:07.478799 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.478798    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-console-config\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 15:50:07.478984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.478808    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-trusted-ca-bundle\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 15:50:07.478984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.478817    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-console-serving-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 15:50:07.478984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.478827    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-service-ca\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 15:50:07.478984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.478835    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-console-oauth-config\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 15:50:07.478984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.478843    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787-oauth-serving-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 15:50:07.478984 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.478842    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fbff8c467-8h4kb"]
Jul 24 15:50:07.482305 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:07.482282    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7fbff8c467-8h4kb"]
Jul 24 15:50:09.226822 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:09.226785    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787" path="/var/lib/kubelet/pods/1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787/volumes"
Jul 24 15:50:21.092281 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:21.092256    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 15:50:21.095480 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:21.095455    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 15:50:21.099696 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:21.099661    2572 kubelet.go:1648] "Image garbage collection succeeded"
Jul 24 15:50:59.748455 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:59.748423    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7wdwz"]
Jul 24 15:50:59.750776 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:59.748663    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787" containerName="console"
Jul 24 15:50:59.750776 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:59.748673    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787" containerName="console"
Jul 24 15:50:59.750776 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:59.748720    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1589bd8c-b8bd-42a5-b12a-4e6d1b9ab787" containerName="console"
Jul 24 15:50:59.751650 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:59.751634    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7wdwz"
Jul 24 15:50:59.754228 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:59.754201    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 24 15:50:59.754349 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:59.754256    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 24 15:50:59.762614 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:59.762594    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7wdwz"]
Jul 24 15:50:59.863769 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:59.863726    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f28631d8-e45e-445a-99f4-17bbccd6abed-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7wdwz\" (UID: \"f28631d8-e45e-445a-99f4-17bbccd6abed\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7wdwz"
Jul 24 15:50:59.863769 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:59.863773    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lr7gj\" (UniqueName: \"kubernetes.io/projected/f28631d8-e45e-445a-99f4-17bbccd6abed-kube-api-access-lr7gj\") pod \"cert-manager-webhook-6486ffd997-7wdwz\" (UID: \"f28631d8-e45e-445a-99f4-17bbccd6abed\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7wdwz"
Jul 24 15:50:59.964187 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:59.964151    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f28631d8-e45e-445a-99f4-17bbccd6abed-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7wdwz\" (UID: \"f28631d8-e45e-445a-99f4-17bbccd6abed\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7wdwz"
Jul 24 15:50:59.964354 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:59.964193    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lr7gj\" (UniqueName: \"kubernetes.io/projected/f28631d8-e45e-445a-99f4-17bbccd6abed-kube-api-access-lr7gj\") pod \"cert-manager-webhook-6486ffd997-7wdwz\" (UID: \"f28631d8-e45e-445a-99f4-17bbccd6abed\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7wdwz"
Jul 24 15:50:59.973366 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:59.973344    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f28631d8-e45e-445a-99f4-17bbccd6abed-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7wdwz\" (UID: \"f28631d8-e45e-445a-99f4-17bbccd6abed\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7wdwz"
Jul 24 15:50:59.973507 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:50:59.973491    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lr7gj\" (UniqueName: \"kubernetes.io/projected/f28631d8-e45e-445a-99f4-17bbccd6abed-kube-api-access-lr7gj\") pod \"cert-manager-webhook-6486ffd997-7wdwz\" (UID: \"f28631d8-e45e-445a-99f4-17bbccd6abed\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7wdwz"
Jul 24 15:51:00.060827 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:00.060725    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7wdwz"
Jul 24 15:51:00.194078 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:00.194050    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7wdwz"]
Jul 24 15:51:00.196415 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:51:00.196387    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf28631d8_e45e_445a_99f4_17bbccd6abed.slice/crio-385acc201a84e4af7dee4c78f2bc96acefec54c5769eb73ede75349c69070542 WatchSource:0}: Error finding container 385acc201a84e4af7dee4c78f2bc96acefec54c5769eb73ede75349c69070542: Status 404 returned error can't find the container with id 385acc201a84e4af7dee4c78f2bc96acefec54c5769eb73ede75349c69070542
Jul 24 15:51:00.198245 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:00.198225    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:51:00.634594 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:00.634557    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7wdwz" event={"ID":"f28631d8-e45e-445a-99f4-17bbccd6abed","Type":"ContainerStarted","Data":"385acc201a84e4af7dee4c78f2bc96acefec54c5769eb73ede75349c69070542"}
Jul 24 15:51:03.647584 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:03.647543    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7wdwz" event={"ID":"f28631d8-e45e-445a-99f4-17bbccd6abed","Type":"ContainerStarted","Data":"7c570c01663b5e4c08c6a3a1d88660860f1d77578b7faad03bc9b6336f091316"}
Jul 24 15:51:03.648119 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:03.647684    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7wdwz"
Jul 24 15:51:03.663411 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:03.663358    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-7wdwz" podStartSLOduration=1.40143404 podStartE2EDuration="4.663342498s" totalImagesPullingTime="3.261908458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:50:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:51:03.6624778 +0000 UTC m=+342.946805407" watchObservedRunningTime="2026-07-24 15:51:03.663342498 +0000 UTC m=+342.947670106"
Jul 24 15:51:07.934308 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:07.934272    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8hr9s"]
Jul 24 15:51:07.937404 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:07.937382    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8hr9s"
Jul 24 15:51:07.939838 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:07.939823    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-85sh2\""
Jul 24 15:51:07.944771 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:07.944748    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8hr9s"]
Jul 24 15:51:08.032612 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:08.032575    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9dqq\" (UniqueName: \"kubernetes.io/projected/f24bfaa9-b63a-4a6b-870e-6f63e0348c07-kube-api-access-s9dqq\") pod \"cert-manager-cainjector-559cdd8c87-8hr9s\" (UID: \"f24bfaa9-b63a-4a6b-870e-6f63e0348c07\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8hr9s"
Jul 24 15:51:08.032778 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:08.032628    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f24bfaa9-b63a-4a6b-870e-6f63e0348c07-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8hr9s\" (UID: \"f24bfaa9-b63a-4a6b-870e-6f63e0348c07\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8hr9s"
Jul 24 15:51:08.133950 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:08.133884    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f24bfaa9-b63a-4a6b-870e-6f63e0348c07-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8hr9s\" (UID: \"f24bfaa9-b63a-4a6b-870e-6f63e0348c07\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8hr9s"
Jul 24 15:51:08.134116 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:08.133977    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9dqq\" (UniqueName: \"kubernetes.io/projected/f24bfaa9-b63a-4a6b-870e-6f63e0348c07-kube-api-access-s9dqq\") pod \"cert-manager-cainjector-559cdd8c87-8hr9s\" (UID: \"f24bfaa9-b63a-4a6b-870e-6f63e0348c07\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8hr9s"
Jul 24 15:51:08.142986 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:08.142952    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f24bfaa9-b63a-4a6b-870e-6f63e0348c07-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8hr9s\" (UID: \"f24bfaa9-b63a-4a6b-870e-6f63e0348c07\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8hr9s"
Jul 24 15:51:08.143134 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:08.143101    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9dqq\" (UniqueName: \"kubernetes.io/projected/f24bfaa9-b63a-4a6b-870e-6f63e0348c07-kube-api-access-s9dqq\") pod \"cert-manager-cainjector-559cdd8c87-8hr9s\" (UID: \"f24bfaa9-b63a-4a6b-870e-6f63e0348c07\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8hr9s"
Jul 24 15:51:08.246888 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:08.246800    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8hr9s"
Jul 24 15:51:08.378419 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:08.378384    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8hr9s"]
Jul 24 15:51:08.381211 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:51:08.381184    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf24bfaa9_b63a_4a6b_870e_6f63e0348c07.slice/crio-d9c978878c0e83661dda3f8d0d87c88234058273f098cdbacc63b0b9129a53e3 WatchSource:0}: Error finding container d9c978878c0e83661dda3f8d0d87c88234058273f098cdbacc63b0b9129a53e3: Status 404 returned error can't find the container with id d9c978878c0e83661dda3f8d0d87c88234058273f098cdbacc63b0b9129a53e3
Jul 24 15:51:08.670840 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:08.670802    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8hr9s" event={"ID":"f24bfaa9-b63a-4a6b-870e-6f63e0348c07","Type":"ContainerStarted","Data":"9a093a040761abeeb35617d17682f7ada96df5eae3514efda9669b4b3b19706f"}
Jul 24 15:51:08.671049 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:08.670849    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8hr9s" event={"ID":"f24bfaa9-b63a-4a6b-870e-6f63e0348c07","Type":"ContainerStarted","Data":"d9c978878c0e83661dda3f8d0d87c88234058273f098cdbacc63b0b9129a53e3"}
Jul 24 15:51:08.686386 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:08.686327    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8hr9s" podStartSLOduration=1.686305854 podStartE2EDuration="1.686305854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:51:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:51:08.685244286 +0000 UTC m=+347.969571905" watchObservedRunningTime="2026-07-24 15:51:08.686305854 +0000 UTC m=+347.970633453"
Jul 24 15:51:09.653386 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:09.653345    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7wdwz"
Jul 24 15:51:32.607384 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.607308    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"]
Jul 24 15:51:32.610460 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.610440    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:32.613576 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.613550    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8dkpp\""
Jul 24 15:51:32.613699 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.613632    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 24 15:51:32.614162 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.614142    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 24 15:51:32.614825 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.614811    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 24 15:51:32.616326 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.616305    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 24 15:51:32.623977 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.623955    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dcf7d71a-81b7-46e6-b7da-b9c04935dab7-manifests\") pod \"opendatahub-operator-controller-manager-5d995667b8-dmnvs\" (UID: \"dcf7d71a-81b7-46e6-b7da-b9c04935dab7\") " pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:32.624077 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.624006    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dcf7d71a-81b7-46e6-b7da-b9c04935dab7-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d995667b8-dmnvs\" (UID: \"dcf7d71a-81b7-46e6-b7da-b9c04935dab7\") " pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:32.624077 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.624040    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdblx\" (UniqueName: \"kubernetes.io/projected/dcf7d71a-81b7-46e6-b7da-b9c04935dab7-kube-api-access-bdblx\") pod \"opendatahub-operator-controller-manager-5d995667b8-dmnvs\" (UID: \"dcf7d71a-81b7-46e6-b7da-b9c04935dab7\") " pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:32.624077 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.624058    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dcf7d71a-81b7-46e6-b7da-b9c04935dab7-tmp\") pod \"opendatahub-operator-controller-manager-5d995667b8-dmnvs\" (UID: \"dcf7d71a-81b7-46e6-b7da-b9c04935dab7\") " pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:32.624193 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.624122    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dcf7d71a-81b7-46e6-b7da-b9c04935dab7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d995667b8-dmnvs\" (UID: \"dcf7d71a-81b7-46e6-b7da-b9c04935dab7\") " pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:32.628057 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.628032    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"]
Jul 24 15:51:32.724732 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.724701    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dcf7d71a-81b7-46e6-b7da-b9c04935dab7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d995667b8-dmnvs\" (UID: \"dcf7d71a-81b7-46e6-b7da-b9c04935dab7\") " pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:32.724908 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.724738    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dcf7d71a-81b7-46e6-b7da-b9c04935dab7-manifests\") pod \"opendatahub-operator-controller-manager-5d995667b8-dmnvs\" (UID: \"dcf7d71a-81b7-46e6-b7da-b9c04935dab7\") " pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:32.724908 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.724769    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dcf7d71a-81b7-46e6-b7da-b9c04935dab7-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d995667b8-dmnvs\" (UID: \"dcf7d71a-81b7-46e6-b7da-b9c04935dab7\") " pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:32.724908 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.724796    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdblx\" (UniqueName: \"kubernetes.io/projected/dcf7d71a-81b7-46e6-b7da-b9c04935dab7-kube-api-access-bdblx\") pod \"opendatahub-operator-controller-manager-5d995667b8-dmnvs\" (UID: \"dcf7d71a-81b7-46e6-b7da-b9c04935dab7\") " pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:32.725106 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.724968    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dcf7d71a-81b7-46e6-b7da-b9c04935dab7-tmp\") pod \"opendatahub-operator-controller-manager-5d995667b8-dmnvs\" (UID: \"dcf7d71a-81b7-46e6-b7da-b9c04935dab7\") " pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:32.725205 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.725177    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/dcf7d71a-81b7-46e6-b7da-b9c04935dab7-manifests\") pod \"opendatahub-operator-controller-manager-5d995667b8-dmnvs\" (UID: \"dcf7d71a-81b7-46e6-b7da-b9c04935dab7\") " pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:32.727505 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.727484    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/dcf7d71a-81b7-46e6-b7da-b9c04935dab7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d995667b8-dmnvs\" (UID: \"dcf7d71a-81b7-46e6-b7da-b9c04935dab7\") " pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:32.727618 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.727506    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dcf7d71a-81b7-46e6-b7da-b9c04935dab7-tmp\") pod \"opendatahub-operator-controller-manager-5d995667b8-dmnvs\" (UID: \"dcf7d71a-81b7-46e6-b7da-b9c04935dab7\") " pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:32.727678 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.727643    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/dcf7d71a-81b7-46e6-b7da-b9c04935dab7-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d995667b8-dmnvs\" (UID: \"dcf7d71a-81b7-46e6-b7da-b9c04935dab7\") " pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:32.733949 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.733900    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdblx\" (UniqueName: \"kubernetes.io/projected/dcf7d71a-81b7-46e6-b7da-b9c04935dab7-kube-api-access-bdblx\") pod \"opendatahub-operator-controller-manager-5d995667b8-dmnvs\" (UID: \"dcf7d71a-81b7-46e6-b7da-b9c04935dab7\") " pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:32.920955 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:32.920896    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:33.057201 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:33.057159    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"]
Jul 24 15:51:33.060094 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:51:33.060061    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcf7d71a_81b7_46e6_b7da_b9c04935dab7.slice/crio-408c34cd6a74629630d1e1a2938d0b52f0e47e3956356db0bfad3bf70e9b11fb WatchSource:0}: Error finding container 408c34cd6a74629630d1e1a2938d0b52f0e47e3956356db0bfad3bf70e9b11fb: Status 404 returned error can't find the container with id 408c34cd6a74629630d1e1a2938d0b52f0e47e3956356db0bfad3bf70e9b11fb
Jul 24 15:51:33.767566 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:33.767519    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs" event={"ID":"dcf7d71a-81b7-46e6-b7da-b9c04935dab7","Type":"ContainerStarted","Data":"408c34cd6a74629630d1e1a2938d0b52f0e47e3956356db0bfad3bf70e9b11fb"}
Jul 24 15:51:36.780903 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:36.780861    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs" event={"ID":"dcf7d71a-81b7-46e6-b7da-b9c04935dab7","Type":"ContainerStarted","Data":"d769457dc1f07f46da12c3fde68799cef4ca6c8d437b75af09712b20678a3f0b"}
Jul 24 15:51:37.785962 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:37.785906    2572 generic.go:356] "Generic (PLEG): container finished" podID="dcf7d71a-81b7-46e6-b7da-b9c04935dab7" containerID="d769457dc1f07f46da12c3fde68799cef4ca6c8d437b75af09712b20678a3f0b" exitCode=0
Jul 24 15:51:37.786407 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:37.785957    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs" event={"ID":"dcf7d71a-81b7-46e6-b7da-b9c04935dab7","Type":"ContainerDied","Data":"d769457dc1f07f46da12c3fde68799cef4ca6c8d437b75af09712b20678a3f0b"}
Jul 24 15:51:38.791282 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:38.791242    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs" event={"ID":"dcf7d71a-81b7-46e6-b7da-b9c04935dab7","Type":"ContainerStarted","Data":"45b606304c293c432eaa03302fb2919ce3aa3a04887fa1fac0d53335a030b728"}
Jul 24 15:51:38.791687 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:38.791355    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:38.815469 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:38.815414    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs" podStartSLOduration=2.284401849 podStartE2EDuration="6.815397031s" totalImagesPullingTime="3.478195011s" totalInitContainerRuntime="1.052800171s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:51:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:51:38.81448894 +0000 UTC m=+378.098816568" watchObservedRunningTime="2026-07-24 15:51:38.815397031 +0000 UTC m=+378.099724640"
Jul 24 15:51:49.798360 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:49.798312    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5d995667b8-dmnvs"
Jul 24 15:51:52.032363 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.032321    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-767449f4cf-4vxsn"]
Jul 24 15:51:52.040212 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.040190    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-767449f4cf-4vxsn"
Jul 24 15:51:52.042961 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.042939    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 24 15:51:52.044000 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.043978    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 24 15:51:52.044124 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.043987    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 24 15:51:52.044256 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.044240    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 24 15:51:52.044329 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.044278    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8vnkm\""
Jul 24 15:51:52.052440 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.052416    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-767449f4cf-4vxsn"]
Jul 24 15:51:52.182853 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.182798    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5bc3e15d-17f4-4dd9-b13f-93730632f09b-tmp\") pod \"kube-auth-proxy-767449f4cf-4vxsn\" (UID: \"5bc3e15d-17f4-4dd9-b13f-93730632f09b\") " pod="openshift-ingress/kube-auth-proxy-767449f4cf-4vxsn"
Jul 24 15:51:52.182853 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.182860    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq2hh\" (UniqueName: \"kubernetes.io/projected/5bc3e15d-17f4-4dd9-b13f-93730632f09b-kube-api-access-zq2hh\") pod \"kube-auth-proxy-767449f4cf-4vxsn\" (UID: \"5bc3e15d-17f4-4dd9-b13f-93730632f09b\") " pod="openshift-ingress/kube-auth-proxy-767449f4cf-4vxsn"
Jul 24 15:51:52.183166 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.183021    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bc3e15d-17f4-4dd9-b13f-93730632f09b-tls-certs\") pod \"kube-auth-proxy-767449f4cf-4vxsn\" (UID: \"5bc3e15d-17f4-4dd9-b13f-93730632f09b\") " pod="openshift-ingress/kube-auth-proxy-767449f4cf-4vxsn"
Jul 24 15:51:52.283495 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.283398    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bc3e15d-17f4-4dd9-b13f-93730632f09b-tls-certs\") pod \"kube-auth-proxy-767449f4cf-4vxsn\" (UID: \"5bc3e15d-17f4-4dd9-b13f-93730632f09b\") " pod="openshift-ingress/kube-auth-proxy-767449f4cf-4vxsn"
Jul 24 15:51:52.283495 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.283448    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5bc3e15d-17f4-4dd9-b13f-93730632f09b-tmp\") pod \"kube-auth-proxy-767449f4cf-4vxsn\" (UID: \"5bc3e15d-17f4-4dd9-b13f-93730632f09b\") " pod="openshift-ingress/kube-auth-proxy-767449f4cf-4vxsn"
Jul 24 15:51:52.283495 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.283476    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq2hh\" (UniqueName: \"kubernetes.io/projected/5bc3e15d-17f4-4dd9-b13f-93730632f09b-kube-api-access-zq2hh\") pod \"kube-auth-proxy-767449f4cf-4vxsn\" (UID: \"5bc3e15d-17f4-4dd9-b13f-93730632f09b\") " pod="openshift-ingress/kube-auth-proxy-767449f4cf-4vxsn"
Jul 24 15:51:52.286040 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.286014    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5bc3e15d-17f4-4dd9-b13f-93730632f09b-tmp\") pod \"kube-auth-proxy-767449f4cf-4vxsn\" (UID: \"5bc3e15d-17f4-4dd9-b13f-93730632f09b\") " pod="openshift-ingress/kube-auth-proxy-767449f4cf-4vxsn"
Jul 24 15:51:52.286279 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.286259    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5bc3e15d-17f4-4dd9-b13f-93730632f09b-tls-certs\") pod \"kube-auth-proxy-767449f4cf-4vxsn\" (UID: \"5bc3e15d-17f4-4dd9-b13f-93730632f09b\") " pod="openshift-ingress/kube-auth-proxy-767449f4cf-4vxsn"
Jul 24 15:51:52.305663 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.305634    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq2hh\" (UniqueName: \"kubernetes.io/projected/5bc3e15d-17f4-4dd9-b13f-93730632f09b-kube-api-access-zq2hh\") pod \"kube-auth-proxy-767449f4cf-4vxsn\" (UID: \"5bc3e15d-17f4-4dd9-b13f-93730632f09b\") " pod="openshift-ingress/kube-auth-proxy-767449f4cf-4vxsn"
Jul 24 15:51:52.353159 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.353124    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-767449f4cf-4vxsn"
Jul 24 15:51:52.494310 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.494285    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-767449f4cf-4vxsn"]
Jul 24 15:51:52.496556 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:51:52.496528    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bc3e15d_17f4_4dd9_b13f_93730632f09b.slice/crio-a17db2f379c5318484bd0ac9df685cef21961525f5c921019c247016618d35ec WatchSource:0}: Error finding container a17db2f379c5318484bd0ac9df685cef21961525f5c921019c247016618d35ec: Status 404 returned error can't find the container with id a17db2f379c5318484bd0ac9df685cef21961525f5c921019c247016618d35ec
Jul 24 15:51:52.842622 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:52.842584    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-767449f4cf-4vxsn" event={"ID":"5bc3e15d-17f4-4dd9-b13f-93730632f09b","Type":"ContainerStarted","Data":"a17db2f379c5318484bd0ac9df685cef21961525f5c921019c247016618d35ec"}
Jul 24 15:51:53.008828 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:53.008784    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-m5lp8"]
Jul 24 15:51:53.013399 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:53.013372    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-m5lp8"
Jul 24 15:51:53.015774 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:53.015752    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 24 15:51:53.025323 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:53.025299    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-m5lp8"]
Jul 24 15:51:53.089968 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:53.089906    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2ltc\" (UniqueName: \"kubernetes.io/projected/c4f5c8cc-3e31-44e5-aff9-ede71e53f010-kube-api-access-l2ltc\") pod \"model-serving-api-54d89bd5fc-m5lp8\" (UID: \"c4f5c8cc-3e31-44e5-aff9-ede71e53f010\") " pod="opendatahub/model-serving-api-54d89bd5fc-m5lp8"
Jul 24 15:51:53.090353 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:53.090043    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4f5c8cc-3e31-44e5-aff9-ede71e53f010-tls-certs\") pod \"model-serving-api-54d89bd5fc-m5lp8\" (UID: \"c4f5c8cc-3e31-44e5-aff9-ede71e53f010\") " pod="opendatahub/model-serving-api-54d89bd5fc-m5lp8"
Jul 24 15:51:53.191405 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:53.191333    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2ltc\" (UniqueName: \"kubernetes.io/projected/c4f5c8cc-3e31-44e5-aff9-ede71e53f010-kube-api-access-l2ltc\") pod \"model-serving-api-54d89bd5fc-m5lp8\" (UID: \"c4f5c8cc-3e31-44e5-aff9-ede71e53f010\") " pod="opendatahub/model-serving-api-54d89bd5fc-m5lp8"
Jul 24 15:51:53.191707 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:53.191670    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4f5c8cc-3e31-44e5-aff9-ede71e53f010-tls-certs\") pod \"model-serving-api-54d89bd5fc-m5lp8\" (UID: \"c4f5c8cc-3e31-44e5-aff9-ede71e53f010\") " pod="opendatahub/model-serving-api-54d89bd5fc-m5lp8"
Jul 24 15:51:53.194825 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:53.194805    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c4f5c8cc-3e31-44e5-aff9-ede71e53f010-tls-certs\") pod \"model-serving-api-54d89bd5fc-m5lp8\" (UID: \"c4f5c8cc-3e31-44e5-aff9-ede71e53f010\") " pod="opendatahub/model-serving-api-54d89bd5fc-m5lp8"
Jul 24 15:51:53.203540 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:53.203515    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2ltc\" (UniqueName: \"kubernetes.io/projected/c4f5c8cc-3e31-44e5-aff9-ede71e53f010-kube-api-access-l2ltc\") pod \"model-serving-api-54d89bd5fc-m5lp8\" (UID: \"c4f5c8cc-3e31-44e5-aff9-ede71e53f010\") " pod="opendatahub/model-serving-api-54d89bd5fc-m5lp8"
Jul 24 15:51:53.323433 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:53.323402    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-m5lp8"
Jul 24 15:51:53.470531 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:53.470502    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-m5lp8"]
Jul 24 15:51:53.472586 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:51:53.472553    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4f5c8cc_3e31_44e5_aff9_ede71e53f010.slice/crio-3fe1eacfef4876da699a1d5dee9e55602dad20525143b77ab2acc5cd18a511c6 WatchSource:0}: Error finding container 3fe1eacfef4876da699a1d5dee9e55602dad20525143b77ab2acc5cd18a511c6: Status 404 returned error can't find the container with id 3fe1eacfef4876da699a1d5dee9e55602dad20525143b77ab2acc5cd18a511c6
Jul 24 15:51:53.848223 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:53.848134    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-m5lp8" event={"ID":"c4f5c8cc-3e31-44e5-aff9-ede71e53f010","Type":"ContainerStarted","Data":"3fe1eacfef4876da699a1d5dee9e55602dad20525143b77ab2acc5cd18a511c6"}
Jul 24 15:51:55.857598 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:55.857502    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-767449f4cf-4vxsn" event={"ID":"5bc3e15d-17f4-4dd9-b13f-93730632f09b","Type":"ContainerStarted","Data":"05aa99f933ca0ee2566ba29de8af20c778eb0a212eb0976287275097318b4b40"}
Jul 24 15:51:55.859067 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:55.859045    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-m5lp8" event={"ID":"c4f5c8cc-3e31-44e5-aff9-ede71e53f010","Type":"ContainerStarted","Data":"d571c62d959cb9fd1548d1f631c6d4105a89d46401c1c49fbab256f5778efdd8"}
Jul 24 15:51:55.859151 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:55.859131    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-m5lp8"
Jul 24 15:51:55.875132 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:55.875090    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-767449f4cf-4vxsn" podStartSLOduration=0.917265707 podStartE2EDuration="3.875074592s" totalImagesPullingTime="2.957808885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:51:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:51:55.873964413 +0000 UTC m=+395.158292067" watchObservedRunningTime="2026-07-24 15:51:55.875074592 +0000 UTC m=+395.159402199"
Jul 24 15:51:55.889857 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:55.889793    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-m5lp8" podStartSLOduration=1.908900714 podStartE2EDuration="3.889771868s" totalImagesPullingTime="1.980871154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:51:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:51:55.889444165 +0000 UTC m=+395.173771786" watchObservedRunningTime="2026-07-24 15:51:55.889771868 +0000 UTC m=+395.174099478"
Jul 24 15:51:59.226895 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.226863    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"]
Jul 24 15:51:59.230268 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.230248    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.233162 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.233140    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-zfjfm\""
Jul 24 15:51:59.233480 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.233461    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 24 15:51:59.249854 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.249826    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"]
Jul 24 15:51:59.344658 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.344619    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcd3b574-df2d-4ba5-a676-9b17478edede-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.344658 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.344677    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcd3b574-df2d-4ba5-a676-9b17478edede-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.344897 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.344737    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcd3b574-df2d-4ba5-a676-9b17478edede-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.344897 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.344769    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcd3b574-df2d-4ba5-a676-9b17478edede-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.344897 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.344790    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjq5n\" (UniqueName: \"kubernetes.io/projected/fcd3b574-df2d-4ba5-a676-9b17478edede-kube-api-access-mjq5n\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.344897 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.344826    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcd3b574-df2d-4ba5-a676-9b17478edede-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.345112 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.344909    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcd3b574-df2d-4ba5-a676-9b17478edede-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.345112 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.344965    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcd3b574-df2d-4ba5-a676-9b17478edede-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.345112 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.344986    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcd3b574-df2d-4ba5-a676-9b17478edede-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.445589 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.445554    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcd3b574-df2d-4ba5-a676-9b17478edede-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.445589 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.445595    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcd3b574-df2d-4ba5-a676-9b17478edede-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.445839 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.445615    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcd3b574-df2d-4ba5-a676-9b17478edede-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.445839 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.445633    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjq5n\" (UniqueName: \"kubernetes.io/projected/fcd3b574-df2d-4ba5-a676-9b17478edede-kube-api-access-mjq5n\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.445839 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.445662    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcd3b574-df2d-4ba5-a676-9b17478edede-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.445839 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.445688    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcd3b574-df2d-4ba5-a676-9b17478edede-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.445839 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.445706    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcd3b574-df2d-4ba5-a676-9b17478edede-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.445839 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.445724    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcd3b574-df2d-4ba5-a676-9b17478edede-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.445839 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.445829    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcd3b574-df2d-4ba5-a676-9b17478edede-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.446214 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.446042    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fcd3b574-df2d-4ba5-a676-9b17478edede-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.446214 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.446082    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcd3b574-df2d-4ba5-a676-9b17478edede-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.446214 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.446125    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fcd3b574-df2d-4ba5-a676-9b17478edede-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.446355 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.446219    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fcd3b574-df2d-4ba5-a676-9b17478edede-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.446414 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.446396    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fcd3b574-df2d-4ba5-a676-9b17478edede-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.448364 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.448341    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fcd3b574-df2d-4ba5-a676-9b17478edede-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.448573 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.448542    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fcd3b574-df2d-4ba5-a676-9b17478edede-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.453878 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.453845    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjq5n\" (UniqueName: \"kubernetes.io/projected/fcd3b574-df2d-4ba5-a676-9b17478edede-kube-api-access-mjq5n\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.454379 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.454349    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fcd3b574-df2d-4ba5-a676-9b17478edede-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25\" (UID: \"fcd3b574-df2d-4ba5-a676-9b17478edede\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.541277 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.541199    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:51:59.679595 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.679569    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"]
Jul 24 15:51:59.681969 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:51:59.681940    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcd3b574_df2d_4ba5_a676_9b17478edede.slice/crio-59298f742212441dfef8113e4bb7135febb2a2ab3ca22c8eb0b9cc69b6479455 WatchSource:0}: Error finding container 59298f742212441dfef8113e4bb7135febb2a2ab3ca22c8eb0b9cc69b6479455: Status 404 returned error can't find the container with id 59298f742212441dfef8113e4bb7135febb2a2ab3ca22c8eb0b9cc69b6479455
Jul 24 15:51:59.876103 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:51:59.876013    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25" event={"ID":"fcd3b574-df2d-4ba5-a676-9b17478edede","Type":"ContainerStarted","Data":"59298f742212441dfef8113e4bb7135febb2a2ab3ca22c8eb0b9cc69b6479455"}
Jul 24 15:52:01.475190 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.475154    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"]
Jul 24 15:52:01.481868 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.481843    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"
Jul 24 15:52:01.484452 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.484427    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 24 15:52:01.485532 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.485506    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 24 15:52:01.485659 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.485564    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 24 15:52:01.485659 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.485576    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-hp7zx\""
Jul 24 15:52:01.485659 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.485516    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 24 15:52:01.485659 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.485616    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 24 15:52:01.489839 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.489813    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"]
Jul 24 15:52:01.562955 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.562892    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9q8k\" (UniqueName: \"kubernetes.io/projected/8b3fe002-5579-4051-8c41-589b1471cab7-kube-api-access-s9q8k\") pod \"lws-controller-manager-57bd9ccdc-rmz4b\" (UID: \"8b3fe002-5579-4051-8c41-589b1471cab7\") " pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"
Jul 24 15:52:01.563132 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.563060    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b3fe002-5579-4051-8c41-589b1471cab7-cert\") pod \"lws-controller-manager-57bd9ccdc-rmz4b\" (UID: \"8b3fe002-5579-4051-8c41-589b1471cab7\") " pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"
Jul 24 15:52:01.563132 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.563104    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8b3fe002-5579-4051-8c41-589b1471cab7-manager-config\") pod \"lws-controller-manager-57bd9ccdc-rmz4b\" (UID: \"8b3fe002-5579-4051-8c41-589b1471cab7\") " pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"
Jul 24 15:52:01.563308 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.563158    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b3fe002-5579-4051-8c41-589b1471cab7-metrics-cert\") pod \"lws-controller-manager-57bd9ccdc-rmz4b\" (UID: \"8b3fe002-5579-4051-8c41-589b1471cab7\") " pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"
Jul 24 15:52:01.664638 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.664599    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b3fe002-5579-4051-8c41-589b1471cab7-cert\") pod \"lws-controller-manager-57bd9ccdc-rmz4b\" (UID: \"8b3fe002-5579-4051-8c41-589b1471cab7\") " pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"
Jul 24 15:52:01.664804 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.664658    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8b3fe002-5579-4051-8c41-589b1471cab7-manager-config\") pod \"lws-controller-manager-57bd9ccdc-rmz4b\" (UID: \"8b3fe002-5579-4051-8c41-589b1471cab7\") " pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"
Jul 24 15:52:01.664804 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.664686    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b3fe002-5579-4051-8c41-589b1471cab7-metrics-cert\") pod \"lws-controller-manager-57bd9ccdc-rmz4b\" (UID: \"8b3fe002-5579-4051-8c41-589b1471cab7\") " pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"
Jul 24 15:52:01.664804 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.664734    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9q8k\" (UniqueName: \"kubernetes.io/projected/8b3fe002-5579-4051-8c41-589b1471cab7-kube-api-access-s9q8k\") pod \"lws-controller-manager-57bd9ccdc-rmz4b\" (UID: \"8b3fe002-5579-4051-8c41-589b1471cab7\") " pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"
Jul 24 15:52:01.665437 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.665407    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8b3fe002-5579-4051-8c41-589b1471cab7-manager-config\") pod \"lws-controller-manager-57bd9ccdc-rmz4b\" (UID: \"8b3fe002-5579-4051-8c41-589b1471cab7\") " pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"
Jul 24 15:52:01.667938 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.667900    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b3fe002-5579-4051-8c41-589b1471cab7-metrics-cert\") pod \"lws-controller-manager-57bd9ccdc-rmz4b\" (UID: \"8b3fe002-5579-4051-8c41-589b1471cab7\") " pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"
Jul 24 15:52:01.668052 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.667975    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b3fe002-5579-4051-8c41-589b1471cab7-cert\") pod \"lws-controller-manager-57bd9ccdc-rmz4b\" (UID: \"8b3fe002-5579-4051-8c41-589b1471cab7\") " pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"
Jul 24 15:52:01.673289 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.673268    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9q8k\" (UniqueName: \"kubernetes.io/projected/8b3fe002-5579-4051-8c41-589b1471cab7-kube-api-access-s9q8k\") pod \"lws-controller-manager-57bd9ccdc-rmz4b\" (UID: \"8b3fe002-5579-4051-8c41-589b1471cab7\") " pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"
Jul 24 15:52:01.794220 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:01.794120    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"
Jul 24 15:52:02.341499 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:02.341458    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 15:52:02.341591 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:02.341541    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 15:52:02.341591 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:02.341581    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 15:52:02.425953 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:02.424582    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"]
Jul 24 15:52:02.433485 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:52:02.433451    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b3fe002_5579_4051_8c41_589b1471cab7.slice/crio-8ba7707c414b54cec88aee1d2daad393769200392dcbfb387745d44d3b68e791 WatchSource:0}: Error finding container 8ba7707c414b54cec88aee1d2daad393769200392dcbfb387745d44d3b68e791: Status 404 returned error can't find the container with id 8ba7707c414b54cec88aee1d2daad393769200392dcbfb387745d44d3b68e791
Jul 24 15:52:02.899620 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:02.899578    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25" event={"ID":"fcd3b574-df2d-4ba5-a676-9b17478edede","Type":"ContainerStarted","Data":"c61a5460f82e6876d4bf3baa854c2a7113fe5048e936c2660f50ad9cd38f3e49"}
Jul 24 15:52:02.900829 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:02.900801    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b" event={"ID":"8b3fe002-5579-4051-8c41-589b1471cab7","Type":"ContainerStarted","Data":"8ba7707c414b54cec88aee1d2daad393769200392dcbfb387745d44d3b68e791"}
Jul 24 15:52:02.947195 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:02.945864    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25" podStartSLOduration=1.288793997 podStartE2EDuration="3.945840772s" totalImagesPullingTime="2.657046775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:51:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:52:02.944472726 +0000 UTC m=+402.228800333" watchObservedRunningTime="2026-07-24 15:52:02.945840772 +0000 UTC m=+402.230168388"
Jul 24 15:52:03.550278 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:03.541861    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:52:03.550278 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:03.548469    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:52:03.907269 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:03.907205    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:52:03.908367 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:03.908339    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25"
Jul 24 15:52:05.919216 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:05.919091    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b" event={"ID":"8b3fe002-5579-4051-8c41-589b1471cab7","Type":"ContainerStarted","Data":"912e00632f7ed9d00a46d3fbac6b8ebf97abab91db05b1323aaf6924cf00aa73"}
Jul 24 15:52:05.919216 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:05.919173    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"
Jul 24 15:52:05.941481 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:05.941426    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b" podStartSLOduration=2.340492292 podStartE2EDuration="4.941406787s" totalImagesPullingTime="2.600914495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:52:05.938943494 +0000 UTC m=+405.223271092" watchObservedRunningTime="2026-07-24 15:52:05.941406787 +0000 UTC m=+405.225734395"
Jul 24 15:52:06.867817 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:06.867764    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-m5lp8"
Jul 24 15:52:17.928845 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:52:17.928811    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-57bd9ccdc-rmz4b"
Jul 24 15:54:31.835704 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:31.835625    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"]
Jul 24 15:54:31.838794 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:31.838774    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:31.841367 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:31.841345    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-pfkhl\""
Jul 24 15:54:31.853007 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:31.852984    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"]
Jul 24 15:54:31.911992 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:31.911961    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6648611-1f32-4df0-8450-5d3283b76622-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:31.911992 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:31.912003    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6648611-1f32-4df0-8450-5d3283b76622-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:31.912241 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:31.912031    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6648611-1f32-4df0-8450-5d3283b76622-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:31.912241 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:31.912088    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6648611-1f32-4df0-8450-5d3283b76622-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:31.912241 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:31.912139    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xjzv\" (UniqueName: \"kubernetes.io/projected/b6648611-1f32-4df0-8450-5d3283b76622-kube-api-access-8xjzv\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:31.912241 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:31.912195    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6648611-1f32-4df0-8450-5d3283b76622-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:31.912241 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:31.912222    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6648611-1f32-4df0-8450-5d3283b76622-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:31.912447 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:31.912249    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6648611-1f32-4df0-8450-5d3283b76622-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:31.912447 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:31.912275    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6648611-1f32-4df0-8450-5d3283b76622-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.013219 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.013188    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xjzv\" (UniqueName: \"kubernetes.io/projected/b6648611-1f32-4df0-8450-5d3283b76622-kube-api-access-8xjzv\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.013404 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.013240    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6648611-1f32-4df0-8450-5d3283b76622-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.013404 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.013264    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6648611-1f32-4df0-8450-5d3283b76622-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.013404 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.013376    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6648611-1f32-4df0-8450-5d3283b76622-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.013522 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.013428    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6648611-1f32-4df0-8450-5d3283b76622-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.013522 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.013457    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6648611-1f32-4df0-8450-5d3283b76622-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.013522 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.013496    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6648611-1f32-4df0-8450-5d3283b76622-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.013665 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.013536    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6648611-1f32-4df0-8450-5d3283b76622-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.013665 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.013651    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6648611-1f32-4df0-8450-5d3283b76622-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.013765 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.013746    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6648611-1f32-4df0-8450-5d3283b76622-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.013856 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.013822    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6648611-1f32-4df0-8450-5d3283b76622-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.013989 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.013900    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6648611-1f32-4df0-8450-5d3283b76622-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.013989 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.013973    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6648611-1f32-4df0-8450-5d3283b76622-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.014244 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.014225    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6648611-1f32-4df0-8450-5d3283b76622-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.015783 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.015764    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6648611-1f32-4df0-8450-5d3283b76622-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.016206 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.016186    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6648611-1f32-4df0-8450-5d3283b76622-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.021985 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.021960    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6648611-1f32-4df0-8450-5d3283b76622-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.022731 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.022707    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xjzv\" (UniqueName: \"kubernetes.io/projected/b6648611-1f32-4df0-8450-5d3283b76622-kube-api-access-8xjzv\") pod \"maas-default-gateway-openshift-default-78bb78ff78-mp2m6\" (UID: \"b6648611-1f32-4df0-8450-5d3283b76622\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.149459 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.149430    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:32.285165 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.285140    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"]
Jul 24 15:54:32.287839 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:54:32.287808    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6648611_1f32_4df0_8450_5d3283b76622.slice/crio-422f89955c78f1107e2fd709f0d301d64083db5a85a4e3af3521dd5253a3b186 WatchSource:0}: Error finding container 422f89955c78f1107e2fd709f0d301d64083db5a85a4e3af3521dd5253a3b186: Status 404 returned error can't find the container with id 422f89955c78f1107e2fd709f0d301d64083db5a85a4e3af3521dd5253a3b186
Jul 24 15:54:32.290402 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.290375    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 15:54:32.290476 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.290440    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 15:54:32.290476 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.290469    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 15:54:32.509298 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.509176    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6" event={"ID":"b6648611-1f32-4df0-8450-5d3283b76622","Type":"ContainerStarted","Data":"b7d2b13c1c163a3a0ff6dd94e5818bc33cb27c660f2dbae1937a914c39a6ddfc"}
Jul 24 15:54:32.509298 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.509248    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6" event={"ID":"b6648611-1f32-4df0-8450-5d3283b76622","Type":"ContainerStarted","Data":"422f89955c78f1107e2fd709f0d301d64083db5a85a4e3af3521dd5253a3b186"}
Jul 24 15:54:32.530954 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:32.530886    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6" podStartSLOduration=1.5308685340000001 podStartE2EDuration="1.530868534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:54:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:54:32.529488444 +0000 UTC m=+551.813816061" watchObservedRunningTime="2026-07-24 15:54:32.530868534 +0000 UTC m=+551.815196198"
Jul 24 15:54:33.149712 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:33.149677    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:33.154093 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:33.154067    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:33.513369 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:33.513284    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:54:33.514192 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:54:33.514171    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:55:02.678673 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.678638    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-966c97988-k47vd"]
Jul 24 15:55:02.681809 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.681791    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-966c97988-k47vd"
Jul 24 15:55:02.684552 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.684530    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 24 15:55:02.685458 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.685439    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-wqb4g\""
Jul 24 15:55:02.685543 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.685488    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 24 15:55:02.689909 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.689889    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-966c97988-k47vd"]
Jul 24 15:55:02.765292 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.765254    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-966c97988-k47vd"]
Jul 24 15:55:02.765532 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:55:02.765511    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-nsw64], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-966c97988-k47vd" podUID="00929d32-0c86-43fb-9ccd-8c8a1c73cb94"
Jul 24 15:55:02.773042 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.773011    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsw64\" (UniqueName: \"kubernetes.io/projected/00929d32-0c86-43fb-9ccd-8c8a1c73cb94-kube-api-access-nsw64\") pod \"authorino-966c97988-k47vd\" (UID: \"00929d32-0c86-43fb-9ccd-8c8a1c73cb94\") " pod="kuadrant-system/authorino-966c97988-k47vd"
Jul 24 15:55:02.785861 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.785807    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7554b85c6d-5jc6v"]
Jul 24 15:55:02.789341 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.789321    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7554b85c6d-5jc6v"
Jul 24 15:55:02.791655 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.791634    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 24 15:55:02.796273 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.796250    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7554b85c6d-5jc6v"]
Jul 24 15:55:02.874236 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.874200    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k64gn\" (UniqueName: \"kubernetes.io/projected/15d869c7-216b-41b0-9243-02199039409c-kube-api-access-k64gn\") pod \"authorino-7554b85c6d-5jc6v\" (UID: \"15d869c7-216b-41b0-9243-02199039409c\") " pod="kuadrant-system/authorino-7554b85c6d-5jc6v"
Jul 24 15:55:02.874419 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.874322    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/15d869c7-216b-41b0-9243-02199039409c-tls-cert\") pod \"authorino-7554b85c6d-5jc6v\" (UID: \"15d869c7-216b-41b0-9243-02199039409c\") " pod="kuadrant-system/authorino-7554b85c6d-5jc6v"
Jul 24 15:55:02.874419 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.874356    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsw64\" (UniqueName: \"kubernetes.io/projected/00929d32-0c86-43fb-9ccd-8c8a1c73cb94-kube-api-access-nsw64\") pod \"authorino-966c97988-k47vd\" (UID: \"00929d32-0c86-43fb-9ccd-8c8a1c73cb94\") " pod="kuadrant-system/authorino-966c97988-k47vd"
Jul 24 15:55:02.883905 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.883869    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsw64\" (UniqueName: \"kubernetes.io/projected/00929d32-0c86-43fb-9ccd-8c8a1c73cb94-kube-api-access-nsw64\") pod \"authorino-966c97988-k47vd\" (UID: \"00929d32-0c86-43fb-9ccd-8c8a1c73cb94\") " pod="kuadrant-system/authorino-966c97988-k47vd"
Jul 24 15:55:02.975224 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.975133    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/15d869c7-216b-41b0-9243-02199039409c-tls-cert\") pod \"authorino-7554b85c6d-5jc6v\" (UID: \"15d869c7-216b-41b0-9243-02199039409c\") " pod="kuadrant-system/authorino-7554b85c6d-5jc6v"
Jul 24 15:55:02.975224 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.975180    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k64gn\" (UniqueName: \"kubernetes.io/projected/15d869c7-216b-41b0-9243-02199039409c-kube-api-access-k64gn\") pod \"authorino-7554b85c6d-5jc6v\" (UID: \"15d869c7-216b-41b0-9243-02199039409c\") " pod="kuadrant-system/authorino-7554b85c6d-5jc6v"
Jul 24 15:55:02.977674 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.977644    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/15d869c7-216b-41b0-9243-02199039409c-tls-cert\") pod \"authorino-7554b85c6d-5jc6v\" (UID: \"15d869c7-216b-41b0-9243-02199039409c\") " pod="kuadrant-system/authorino-7554b85c6d-5jc6v"
Jul 24 15:55:02.983864 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:02.983845    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k64gn\" (UniqueName: \"kubernetes.io/projected/15d869c7-216b-41b0-9243-02199039409c-kube-api-access-k64gn\") pod \"authorino-7554b85c6d-5jc6v\" (UID: \"15d869c7-216b-41b0-9243-02199039409c\") " pod="kuadrant-system/authorino-7554b85c6d-5jc6v"
Jul 24 15:55:03.098367 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:03.098334    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7554b85c6d-5jc6v"
Jul 24 15:55:03.230123 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:03.230098    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7554b85c6d-5jc6v"]
Jul 24 15:55:03.232061 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:55:03.232036    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15d869c7_216b_41b0_9243_02199039409c.slice/crio-cc73d68085428435d47ff1c78cd8aff907eaddb59230d851dd00477a84c073cd WatchSource:0}: Error finding container cc73d68085428435d47ff1c78cd8aff907eaddb59230d851dd00477a84c073cd: Status 404 returned error can't find the container with id cc73d68085428435d47ff1c78cd8aff907eaddb59230d851dd00477a84c073cd
Jul 24 15:55:03.634109 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:03.634024    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7554b85c6d-5jc6v" event={"ID":"15d869c7-216b-41b0-9243-02199039409c","Type":"ContainerStarted","Data":"cc73d68085428435d47ff1c78cd8aff907eaddb59230d851dd00477a84c073cd"}
Jul 24 15:55:03.634109 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:03.634042    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-966c97988-k47vd"
Jul 24 15:55:03.641016 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:03.640993    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-966c97988-k47vd"
Jul 24 15:55:03.782768 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:03.782732    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsw64\" (UniqueName: \"kubernetes.io/projected/00929d32-0c86-43fb-9ccd-8c8a1c73cb94-kube-api-access-nsw64\") pod \"00929d32-0c86-43fb-9ccd-8c8a1c73cb94\" (UID: \"00929d32-0c86-43fb-9ccd-8c8a1c73cb94\") "
Jul 24 15:55:03.786123 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:03.786078    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00929d32-0c86-43fb-9ccd-8c8a1c73cb94-kube-api-access-nsw64" (OuterVolumeSpecName: "kube-api-access-nsw64") pod "00929d32-0c86-43fb-9ccd-8c8a1c73cb94" (UID: "00929d32-0c86-43fb-9ccd-8c8a1c73cb94"). InnerVolumeSpecName "kube-api-access-nsw64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:55:03.884686 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:03.884597    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsw64\" (UniqueName: \"kubernetes.io/projected/00929d32-0c86-43fb-9ccd-8c8a1c73cb94-kube-api-access-nsw64\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 15:55:04.640139 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:04.640110    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-966c97988-k47vd"
Jul 24 15:55:04.673970 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:04.673905    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-966c97988-k47vd"]
Jul 24 15:55:04.680246 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:04.680217    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-966c97988-k47vd"]
Jul 24 15:55:05.225239 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:05.225202    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00929d32-0c86-43fb-9ccd-8c8a1c73cb94" path="/var/lib/kubelet/pods/00929d32-0c86-43fb-9ccd-8c8a1c73cb94/volumes"
Jul 24 15:55:06.652624 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:06.652556    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7554b85c6d-5jc6v" event={"ID":"15d869c7-216b-41b0-9243-02199039409c","Type":"ContainerStarted","Data":"73a21927ed6b2ca5f315f4045bbc6d71581c18e787a992a5ff3208b4bc0f3ab8"}
Jul 24 15:55:06.674449 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:06.674384    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7554b85c6d-5jc6v" podStartSLOduration=1.805052035 podStartE2EDuration="4.674367257s" totalImagesPullingTime="2.869315222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:55:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:55:06.672430357 +0000 UTC m=+585.956757969" watchObservedRunningTime="2026-07-24 15:55:06.674367257 +0000 UTC m=+585.958694902"
Jul 24 15:55:21.123201 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:21.123164    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 15:55:21.124473 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:21.124451    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 15:55:55.860876 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:55.860761    2572 generic.go:356] "Generic (PLEG): container finished" podID="b6648611-1f32-4df0-8450-5d3283b76622" containerID="b7d2b13c1c163a3a0ff6dd94e5818bc33cb27c660f2dbae1937a914c39a6ddfc" exitCode=0
Jul 24 15:55:55.860876 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:55.860839    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6" event={"ID":"b6648611-1f32-4df0-8450-5d3283b76622","Type":"ContainerDied","Data":"b7d2b13c1c163a3a0ff6dd94e5818bc33cb27c660f2dbae1937a914c39a6ddfc"}
Jul 24 15:55:55.861464 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:55.861369    2572 scope.go:117] "RemoveContainer" containerID="b7d2b13c1c163a3a0ff6dd94e5818bc33cb27c660f2dbae1937a914c39a6ddfc"
Jul 24 15:55:55.862398 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:55.862372    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 15:55:55.862504 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:55.862440    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 15:55:55.862504 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:55.862474    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 15:55:56.867394 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:56.867358    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6" event={"ID":"b6648611-1f32-4df0-8450-5d3283b76622","Type":"ContainerStarted","Data":"b316f983f65ebcff7551a00b565e25824dd4b8d036dd7b0068dc6d424a545849"}
Jul 24 15:55:57.150178 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:57.150137    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:55:57.150447 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:57.150424    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-mp2m6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" start-of-body=
Jul 24 15:55:57.150502 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:57.150481    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6" podUID="b6648611-1f32-4df0-8450-5d3283b76622" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused"
Jul 24 15:55:58.150899 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:58.150853    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-mp2m6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" start-of-body=
Jul 24 15:55:58.151344 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:58.150949    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6" podUID="b6648611-1f32-4df0-8450-5d3283b76622" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused"
Jul 24 15:55:59.149865 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:59.149826    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-mp2m6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" start-of-body=
Jul 24 15:55:59.150077 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:55:59.149937    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6" podUID="b6648611-1f32-4df0-8450-5d3283b76622" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused"
Jul 24 15:56:00.150509 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:00.150469    2572 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-mp2m6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused" start-of-body=
Jul 24 15:56:00.150911 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:00.150532    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6" podUID="b6648611-1f32-4df0-8450-5d3283b76622" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.24:15021/healthz/ready\": dial tcp 10.132.0.24:15021: connect: connection refused"
Jul 24 15:56:01.153825 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:01.153798    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:56:01.154228 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:01.154131    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:56:01.154883 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:01.154863    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-mp2m6"
Jul 24 15:56:39.694474 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.694432    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"]
Jul 24 15:56:39.702118 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.702086    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:39.706572 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.705965    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ngjlc\""
Jul 24 15:56:39.706572 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.705987    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 24 15:56:39.706572 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.706446    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 24 15:56:39.706998 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.706620    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 24 15:56:39.709966 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.709943    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"]
Jul 24 15:56:39.784621 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.784579    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkcgs\" (UniqueName: \"kubernetes.io/projected/43454aa6-3fdf-4d75-b335-3f89d28271f6-kube-api-access-mkcgs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj\" (UID: \"43454aa6-3fdf-4d75-b335-3f89d28271f6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:39.784621 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.784627    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43454aa6-3fdf-4d75-b335-3f89d28271f6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj\" (UID: \"43454aa6-3fdf-4d75-b335-3f89d28271f6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:39.784842 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.784656    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43454aa6-3fdf-4d75-b335-3f89d28271f6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj\" (UID: \"43454aa6-3fdf-4d75-b335-3f89d28271f6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:39.784842 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.784689    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43454aa6-3fdf-4d75-b335-3f89d28271f6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj\" (UID: \"43454aa6-3fdf-4d75-b335-3f89d28271f6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:39.784842 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.784711    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43454aa6-3fdf-4d75-b335-3f89d28271f6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj\" (UID: \"43454aa6-3fdf-4d75-b335-3f89d28271f6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:39.784842 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.784790    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43454aa6-3fdf-4d75-b335-3f89d28271f6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj\" (UID: \"43454aa6-3fdf-4d75-b335-3f89d28271f6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:39.885560 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.885491    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43454aa6-3fdf-4d75-b335-3f89d28271f6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj\" (UID: \"43454aa6-3fdf-4d75-b335-3f89d28271f6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:39.885762 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.885585    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkcgs\" (UniqueName: \"kubernetes.io/projected/43454aa6-3fdf-4d75-b335-3f89d28271f6-kube-api-access-mkcgs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj\" (UID: \"43454aa6-3fdf-4d75-b335-3f89d28271f6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:39.885762 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.885620    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43454aa6-3fdf-4d75-b335-3f89d28271f6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj\" (UID: \"43454aa6-3fdf-4d75-b335-3f89d28271f6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:39.885762 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.885645    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43454aa6-3fdf-4d75-b335-3f89d28271f6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj\" (UID: \"43454aa6-3fdf-4d75-b335-3f89d28271f6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:39.885762 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.885677    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43454aa6-3fdf-4d75-b335-3f89d28271f6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj\" (UID: \"43454aa6-3fdf-4d75-b335-3f89d28271f6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:39.885762 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.885711    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43454aa6-3fdf-4d75-b335-3f89d28271f6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj\" (UID: \"43454aa6-3fdf-4d75-b335-3f89d28271f6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:39.886083 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.885971    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43454aa6-3fdf-4d75-b335-3f89d28271f6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj\" (UID: \"43454aa6-3fdf-4d75-b335-3f89d28271f6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:39.886083 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.886019    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43454aa6-3fdf-4d75-b335-3f89d28271f6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj\" (UID: \"43454aa6-3fdf-4d75-b335-3f89d28271f6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:39.886180 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.886121    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43454aa6-3fdf-4d75-b335-3f89d28271f6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj\" (UID: \"43454aa6-3fdf-4d75-b335-3f89d28271f6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:39.888191 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.888173    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43454aa6-3fdf-4d75-b335-3f89d28271f6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj\" (UID: \"43454aa6-3fdf-4d75-b335-3f89d28271f6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:39.888292 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.888276    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43454aa6-3fdf-4d75-b335-3f89d28271f6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj\" (UID: \"43454aa6-3fdf-4d75-b335-3f89d28271f6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:39.894357 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:39.894331    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkcgs\" (UniqueName: \"kubernetes.io/projected/43454aa6-3fdf-4d75-b335-3f89d28271f6-kube-api-access-mkcgs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj\" (UID: \"43454aa6-3fdf-4d75-b335-3f89d28271f6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:40.014388 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:40.014299    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:40.148085 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:40.148059    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"]
Jul 24 15:56:40.150765 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:56:40.150740    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43454aa6_3fdf_4d75_b335_3f89d28271f6.slice/crio-232b125d64a4787bbbd9f66ed269f7150c7cae986b79d20dae47e3c21792094b WatchSource:0}: Error finding container 232b125d64a4787bbbd9f66ed269f7150c7cae986b79d20dae47e3c21792094b: Status 404 returned error can't find the container with id 232b125d64a4787bbbd9f66ed269f7150c7cae986b79d20dae47e3c21792094b
Jul 24 15:56:40.152050 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:40.152033    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 15:56:41.059266 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:41.059219    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj" event={"ID":"43454aa6-3fdf-4d75-b335-3f89d28271f6","Type":"ContainerStarted","Data":"232b125d64a4787bbbd9f66ed269f7150c7cae986b79d20dae47e3c21792094b"}
Jul 24 15:56:42.068169 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:42.068121    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj" event={"ID":"43454aa6-3fdf-4d75-b335-3f89d28271f6","Type":"ContainerStarted","Data":"debb44dd74d4c905c23d643c418f7e729fe4b7281d692d3f53b388fa7f5c19e4"}
Jul 24 15:56:42.088157 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:42.088090    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj" podStartSLOduration=1.3511150029999999 podStartE2EDuration="3.088067201s" totalImagesPullingTime="1.736952198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:56:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:56:42.0858486 +0000 UTC m=+681.370176211" watchObservedRunningTime="2026-07-24 15:56:42.088067201 +0000 UTC m=+681.372394810"
Jul 24 15:56:50.015283 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:50.015238    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:50.015877 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:50.015314    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:50.028214 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:50.028189    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:56:50.114468 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:56:50.114433    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj"
Jul 24 15:57:04.917013 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:04.916977    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"]
Jul 24 15:57:04.920195 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:04.920178    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:04.930363 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:04.930344    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 24 15:57:04.942083 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:04.942055    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"]
Jul 24 15:57:05.004948 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.004880    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cd82046-ac5f-46f2-817f-985998cc08f4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x7lls\" (UID: \"8cd82046-ac5f-46f2-817f-985998cc08f4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.005120 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.004957    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cd82046-ac5f-46f2-817f-985998cc08f4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x7lls\" (UID: \"8cd82046-ac5f-46f2-817f-985998cc08f4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.005120 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.004996    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cd82046-ac5f-46f2-817f-985998cc08f4-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x7lls\" (UID: \"8cd82046-ac5f-46f2-817f-985998cc08f4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.005120 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.005032    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cd82046-ac5f-46f2-817f-985998cc08f4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x7lls\" (UID: \"8cd82046-ac5f-46f2-817f-985998cc08f4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.005120 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.005063    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll6d9\" (UniqueName: \"kubernetes.io/projected/8cd82046-ac5f-46f2-817f-985998cc08f4-kube-api-access-ll6d9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x7lls\" (UID: \"8cd82046-ac5f-46f2-817f-985998cc08f4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.005251 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.005136    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cd82046-ac5f-46f2-817f-985998cc08f4-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x7lls\" (UID: \"8cd82046-ac5f-46f2-817f-985998cc08f4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.106464 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.106425    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cd82046-ac5f-46f2-817f-985998cc08f4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x7lls\" (UID: \"8cd82046-ac5f-46f2-817f-985998cc08f4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.106464 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.106461    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cd82046-ac5f-46f2-817f-985998cc08f4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x7lls\" (UID: \"8cd82046-ac5f-46f2-817f-985998cc08f4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.106722 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.106489    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cd82046-ac5f-46f2-817f-985998cc08f4-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x7lls\" (UID: \"8cd82046-ac5f-46f2-817f-985998cc08f4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.106722 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.106607    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cd82046-ac5f-46f2-817f-985998cc08f4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x7lls\" (UID: \"8cd82046-ac5f-46f2-817f-985998cc08f4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.106722 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.106656    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll6d9\" (UniqueName: \"kubernetes.io/projected/8cd82046-ac5f-46f2-817f-985998cc08f4-kube-api-access-ll6d9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x7lls\" (UID: \"8cd82046-ac5f-46f2-817f-985998cc08f4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.106722 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.106679    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cd82046-ac5f-46f2-817f-985998cc08f4-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x7lls\" (UID: \"8cd82046-ac5f-46f2-817f-985998cc08f4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.106964 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.106885    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cd82046-ac5f-46f2-817f-985998cc08f4-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x7lls\" (UID: \"8cd82046-ac5f-46f2-817f-985998cc08f4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.107024 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.106970    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cd82046-ac5f-46f2-817f-985998cc08f4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x7lls\" (UID: \"8cd82046-ac5f-46f2-817f-985998cc08f4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.107024 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.106995    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cd82046-ac5f-46f2-817f-985998cc08f4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x7lls\" (UID: \"8cd82046-ac5f-46f2-817f-985998cc08f4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.108946 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.108908    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cd82046-ac5f-46f2-817f-985998cc08f4-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x7lls\" (UID: \"8cd82046-ac5f-46f2-817f-985998cc08f4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.109255 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.109240    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cd82046-ac5f-46f2-817f-985998cc08f4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x7lls\" (UID: \"8cd82046-ac5f-46f2-817f-985998cc08f4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.114811 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.114784    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll6d9\" (UniqueName: \"kubernetes.io/projected/8cd82046-ac5f-46f2-817f-985998cc08f4-kube-api-access-ll6d9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-x7lls\" (UID: \"8cd82046-ac5f-46f2-817f-985998cc08f4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.229466 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.229427    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:05.362880 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:05.362851    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"]
Jul 24 15:57:05.365145 ip-10-0-133-124 kubenswrapper[2572]: W0724 15:57:05.365116    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8cd82046_ac5f_46f2_817f_985998cc08f4.slice/crio-a82078cb944cac26efeee84d1d4c01eaa77278d51aa7b2770907b7612376a09b WatchSource:0}: Error finding container a82078cb944cac26efeee84d1d4c01eaa77278d51aa7b2770907b7612376a09b: Status 404 returned error can't find the container with id a82078cb944cac26efeee84d1d4c01eaa77278d51aa7b2770907b7612376a09b
Jul 24 15:57:06.169595 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:06.169561    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls" event={"ID":"8cd82046-ac5f-46f2-817f-985998cc08f4","Type":"ContainerStarted","Data":"a82078cb944cac26efeee84d1d4c01eaa77278d51aa7b2770907b7612376a09b"}
Jul 24 15:57:07.175648 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:07.175606    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls" event={"ID":"8cd82046-ac5f-46f2-817f-985998cc08f4","Type":"ContainerStarted","Data":"6d973f3c3fb7e4c6053e06c899f56f9b4e672d7bded9cdfa41b2ef344ce85b01"}
Jul 24 15:57:07.193316 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:07.193264    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls" podStartSLOduration=1.5925826330000001 podStartE2EDuration="3.19324749s" totalImagesPullingTime="1.600664857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 15:57:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 15:57:07.192382116 +0000 UTC m=+706.476709747" watchObservedRunningTime="2026-07-24 15:57:07.19324749 +0000 UTC m=+706.477575098"
Jul 24 15:57:15.230555 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:15.230477    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:15.230555 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:15.230522    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:15.243106 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:15.243080    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:57:16.225782 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:57:16.225750    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-x7lls"
Jul 24 15:58:03.132093 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:03.132055    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7554b85c6d-5jc6v"]
Jul 24 15:58:03.132674 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:03.132277    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7554b85c6d-5jc6v" podUID="15d869c7-216b-41b0-9243-02199039409c" containerName="authorino" containerID="cri-o://73a21927ed6b2ca5f315f4045bbc6d71581c18e787a992a5ff3208b4bc0f3ab8" gracePeriod=30
Jul 24 15:58:03.379518 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:03.379493    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7554b85c6d-5jc6v"
Jul 24 15:58:03.405102 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:03.405068    2572 generic.go:356] "Generic (PLEG): container finished" podID="15d869c7-216b-41b0-9243-02199039409c" containerID="73a21927ed6b2ca5f315f4045bbc6d71581c18e787a992a5ff3208b4bc0f3ab8" exitCode=0
Jul 24 15:58:03.405268 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:03.405117    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7554b85c6d-5jc6v"
Jul 24 15:58:03.405268 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:03.405153    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7554b85c6d-5jc6v" event={"ID":"15d869c7-216b-41b0-9243-02199039409c","Type":"ContainerDied","Data":"73a21927ed6b2ca5f315f4045bbc6d71581c18e787a992a5ff3208b4bc0f3ab8"}
Jul 24 15:58:03.405268 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:03.405221    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7554b85c6d-5jc6v" event={"ID":"15d869c7-216b-41b0-9243-02199039409c","Type":"ContainerDied","Data":"cc73d68085428435d47ff1c78cd8aff907eaddb59230d851dd00477a84c073cd"}
Jul 24 15:58:03.405268 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:03.405246    2572 scope.go:117] "RemoveContainer" containerID="73a21927ed6b2ca5f315f4045bbc6d71581c18e787a992a5ff3208b4bc0f3ab8"
Jul 24 15:58:03.413964 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:03.413909    2572 scope.go:117] "RemoveContainer" containerID="73a21927ed6b2ca5f315f4045bbc6d71581c18e787a992a5ff3208b4bc0f3ab8"
Jul 24 15:58:03.414254 ip-10-0-133-124 kubenswrapper[2572]: E0724 15:58:03.414228    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73a21927ed6b2ca5f315f4045bbc6d71581c18e787a992a5ff3208b4bc0f3ab8\": container with ID starting with 73a21927ed6b2ca5f315f4045bbc6d71581c18e787a992a5ff3208b4bc0f3ab8 not found: ID does not exist" containerID="73a21927ed6b2ca5f315f4045bbc6d71581c18e787a992a5ff3208b4bc0f3ab8"
Jul 24 15:58:03.414316 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:03.414257    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73a21927ed6b2ca5f315f4045bbc6d71581c18e787a992a5ff3208b4bc0f3ab8"} err="failed to get container status \"73a21927ed6b2ca5f315f4045bbc6d71581c18e787a992a5ff3208b4bc0f3ab8\": rpc error: code = NotFound desc = could not find container \"73a21927ed6b2ca5f315f4045bbc6d71581c18e787a992a5ff3208b4bc0f3ab8\": container with ID starting with 73a21927ed6b2ca5f315f4045bbc6d71581c18e787a992a5ff3208b4bc0f3ab8 not found: ID does not exist"
Jul 24 15:58:03.449282 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:03.449259    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k64gn\" (UniqueName: \"kubernetes.io/projected/15d869c7-216b-41b0-9243-02199039409c-kube-api-access-k64gn\") pod \"15d869c7-216b-41b0-9243-02199039409c\" (UID: \"15d869c7-216b-41b0-9243-02199039409c\") "
Jul 24 15:58:03.449401 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:03.449353    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/15d869c7-216b-41b0-9243-02199039409c-tls-cert\") pod \"15d869c7-216b-41b0-9243-02199039409c\" (UID: \"15d869c7-216b-41b0-9243-02199039409c\") "
Jul 24 15:58:03.451839 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:03.451809    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15d869c7-216b-41b0-9243-02199039409c-kube-api-access-k64gn" (OuterVolumeSpecName: "kube-api-access-k64gn") pod "15d869c7-216b-41b0-9243-02199039409c" (UID: "15d869c7-216b-41b0-9243-02199039409c"). InnerVolumeSpecName "kube-api-access-k64gn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 15:58:03.459208 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:03.459184    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15d869c7-216b-41b0-9243-02199039409c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "15d869c7-216b-41b0-9243-02199039409c" (UID: "15d869c7-216b-41b0-9243-02199039409c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 15:58:03.549957 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:03.549900    2572 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/15d869c7-216b-41b0-9243-02199039409c-tls-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 15:58:03.549957 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:03.549955    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k64gn\" (UniqueName: \"kubernetes.io/projected/15d869c7-216b-41b0-9243-02199039409c-kube-api-access-k64gn\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 15:58:03.725284 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:03.725254    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7554b85c6d-5jc6v"]
Jul 24 15:58:03.730170 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:03.730138    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7554b85c6d-5jc6v"]
Jul 24 15:58:05.223443 ip-10-0-133-124 kubenswrapper[2572]: I0724 15:58:05.223413    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15d869c7-216b-41b0-9243-02199039409c" path="/var/lib/kubelet/pods/15d869c7-216b-41b0-9243-02199039409c/volumes"
Jul 24 16:00:00.145787 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:00:00.145750    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748480-pfw2x"]
Jul 24 16:00:00.146282 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:00:00.146120    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d869c7-216b-41b0-9243-02199039409c" containerName="authorino"
Jul 24 16:00:00.146282 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:00:00.146132    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d869c7-216b-41b0-9243-02199039409c" containerName="authorino"
Jul 24 16:00:00.146282 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:00:00.146203    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="15d869c7-216b-41b0-9243-02199039409c" containerName="authorino"
Jul 24 16:00:00.148958 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:00:00.148942    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-pfw2x"
Jul 24 16:00:00.151413 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:00:00.151389    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kv2w9\""
Jul 24 16:00:00.157058 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:00:00.157032    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748480-pfw2x"]
Jul 24 16:00:00.159632 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:00:00.159610    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-pfw2x"
Jul 24 16:00:00.293390 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:00:00.293364    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748480-pfw2x"]
Jul 24 16:00:00.295894 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:00:00.295867    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ae7a9e6_71cd_41b8_ab6a_cf2f3248e607.slice/crio-17825daf9f8f9804a82190646e572b9a1609f61987118567e8b010569aa2ab5b WatchSource:0}: Error finding container 17825daf9f8f9804a82190646e572b9a1609f61987118567e8b010569aa2ab5b: Status 404 returned error can't find the container with id 17825daf9f8f9804a82190646e572b9a1609f61987118567e8b010569aa2ab5b
Jul 24 16:00:00.903293 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:00:00.903255    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-pfw2x" event={"ID":"7ae7a9e6-71cd-41b8-ab6a-cf2f3248e607","Type":"ContainerStarted","Data":"17825daf9f8f9804a82190646e572b9a1609f61987118567e8b010569aa2ab5b"}
Jul 24 16:00:03.919849 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:00:03.919812    2572 generic.go:356] "Generic (PLEG): container finished" podID="7ae7a9e6-71cd-41b8-ab6a-cf2f3248e607" containerID="0f54c1ee25fc666bf687db3464d3df39cf624933837ffecfd532b016ee374e77" exitCode=0
Jul 24 16:00:03.920246 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:00:03.919897    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-pfw2x" event={"ID":"7ae7a9e6-71cd-41b8-ab6a-cf2f3248e607","Type":"ContainerDied","Data":"0f54c1ee25fc666bf687db3464d3df39cf624933837ffecfd532b016ee374e77"}
Jul 24 16:00:05.056657 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:00:05.056628    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-pfw2x"
Jul 24 16:00:05.928940 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:00:05.928887    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-pfw2x" event={"ID":"7ae7a9e6-71cd-41b8-ab6a-cf2f3248e607","Type":"ContainerDied","Data":"17825daf9f8f9804a82190646e572b9a1609f61987118567e8b010569aa2ab5b"}
Jul 24 16:00:05.928940 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:00:05.928908    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748480-pfw2x"
Jul 24 16:00:05.929157 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:00:05.928954    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17825daf9f8f9804a82190646e572b9a1609f61987118567e8b010569aa2ab5b"
Jul 24 16:00:21.147686 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:00:21.147605    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 16:00:21.148181 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:00:21.147880    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 16:05:21.171436 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:05:21.171401    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 16:05:21.172353 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:05:21.172334    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 16:08:43.200168 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.200134    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"]
Jul 24 16:08:43.200642 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.200461    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ae7a9e6-71cd-41b8-ab6a-cf2f3248e607" containerName="cleanup"
Jul 24 16:08:43.200642 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.200472    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ae7a9e6-71cd-41b8-ab6a-cf2f3248e607" containerName="cleanup"
Jul 24 16:08:43.200642 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.200543    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ae7a9e6-71cd-41b8-ab6a-cf2f3248e607" containerName="cleanup"
Jul 24 16:08:43.203285 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.203268    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.205958 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.205930    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-0d2b906a-kserve-self-signed-certs\""
Jul 24 16:08:43.224788 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.224763    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"]
Jul 24 16:08:43.308087 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.308046    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-home\") pod \"e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") " pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.308267 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.308121    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-model-cache\") pod \"e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") " pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.308267 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.308203    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87zgk\" (UniqueName: \"kubernetes.io/projected/ffa485bf-0cac-447b-bda4-7c71d76c72df-kube-api-access-87zgk\") pod \"e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") " pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.308267 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.308255    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffa485bf-0cac-447b-bda4-7c71d76c72df-tls-certs\") pod \"e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") " pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.308420 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.308312    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-dshm\") pod \"e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") " pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.308420 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.308338    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-tmp-dir\") pod \"e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") " pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.409522 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.409486    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87zgk\" (UniqueName: \"kubernetes.io/projected/ffa485bf-0cac-447b-bda4-7c71d76c72df-kube-api-access-87zgk\") pod \"e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") " pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.409711 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.409529    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffa485bf-0cac-447b-bda4-7c71d76c72df-tls-certs\") pod \"e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") " pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.409711 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.409565    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-dshm\") pod \"e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") " pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.409711 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.409586    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-tmp-dir\") pod \"e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") " pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.409711 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.409639    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-home\") pod \"e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") " pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.409711 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.409695    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-model-cache\") pod \"e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") " pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.410107 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.410085    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-tmp-dir\") pod \"e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") " pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.410184 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.410121    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-home\") pod \"e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") " pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.410223 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.410185    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-model-cache\") pod \"e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") " pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.412089 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.412071    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-dshm\") pod \"e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") " pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.412253 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.412237    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffa485bf-0cac-447b-bda4-7c71d76c72df-tls-certs\") pod \"e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") " pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.420035 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.420012    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87zgk\" (UniqueName: \"kubernetes.io/projected/ffa485bf-0cac-447b-bda4-7c71d76c72df-kube-api-access-87zgk\") pod \"e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") " pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.511983 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.511871    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:43.646933 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:43.646886    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"]
Jul 24 16:08:43.649112 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:08:43.649087    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podffa485bf_0cac_447b_bda4_7c71d76c72df.slice/crio-2e8cfa06feddada4d71e9d3f1791f3af4536f32cd58e869cd0d0288c6b5592a1 WatchSource:0}: Error finding container 2e8cfa06feddada4d71e9d3f1791f3af4536f32cd58e869cd0d0288c6b5592a1: Status 404 returned error can't find the container with id 2e8cfa06feddada4d71e9d3f1791f3af4536f32cd58e869cd0d0288c6b5592a1
Jul 24 16:08:44.159789 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:44.159744    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq" event={"ID":"ffa485bf-0cac-447b-bda4-7c71d76c72df","Type":"ContainerStarted","Data":"271e0ac6a245d42ceaa6161678d9a460f77def903d33faeae1ad94497e9d027e"}
Jul 24 16:08:44.160002 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:44.159800    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq" event={"ID":"ffa485bf-0cac-447b-bda4-7c71d76c72df","Type":"ContainerStarted","Data":"2e8cfa06feddada4d71e9d3f1791f3af4536f32cd58e869cd0d0288c6b5592a1"}
Jul 24 16:08:44.177566 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:44.177518    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq" podStartSLOduration=1.177500851 podStartE2EDuration="1.177500851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:08:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:08:44.176638884 +0000 UTC m=+1403.460966492" watchObservedRunningTime="2026-07-24 16:08:44.177500851 +0000 UTC m=+1403.461828459"
Jul 24 16:08:53.513119 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:53.513020    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:53.513119 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:53.513073    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:53.526182 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:53.526151    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:54.213964 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:54.213936    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:08:59.383595 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.383558    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"]
Jul 24 16:08:59.387128 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.387110    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.389507 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.389485    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-0d2b906a-kserve-self-signed-certs\""
Jul 24 16:08:59.397422 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.397398    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"]
Jul 24 16:08:59.447277 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.447247    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-dshm\") pod \"e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") " pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.447440 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.447285    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb22bacf-0695-43f1-8882-5703cd460d44-tls-certs\") pod \"e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") " pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.447440 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.447318    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-tmp-dir\") pod \"e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") " pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.447440 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.447378    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j94q\" (UniqueName: \"kubernetes.io/projected/cb22bacf-0695-43f1-8882-5703cd460d44-kube-api-access-7j94q\") pod \"e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") " pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.447440 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.447424    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-model-cache\") pod \"e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") " pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.447605 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.447449    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-home\") pod \"e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") " pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.548748 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.548713    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-dshm\") pod \"e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") " pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.548748 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.548751    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb22bacf-0695-43f1-8882-5703cd460d44-tls-certs\") pod \"e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") " pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.549028 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.548783    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-tmp-dir\") pod \"e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") " pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.549028 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.548813    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j94q\" (UniqueName: \"kubernetes.io/projected/cb22bacf-0695-43f1-8882-5703cd460d44-kube-api-access-7j94q\") pod \"e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") " pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.549028 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.548852    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-model-cache\") pod \"e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") " pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.549028 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.548881    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-home\") pod \"e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") " pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.549256 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.549223    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-tmp-dir\") pod \"e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") " pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.549311 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.549294    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-model-cache\") pod \"e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") " pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.549348 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.549330    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-home\") pod \"e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") " pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.551292 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.551271    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-dshm\") pod \"e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") " pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.551461 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.551444    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb22bacf-0695-43f1-8882-5703cd460d44-tls-certs\") pod \"e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") " pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.558269 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.558246    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j94q\" (UniqueName: \"kubernetes.io/projected/cb22bacf-0695-43f1-8882-5703cd460d44-kube-api-access-7j94q\") pod \"e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") " pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.697351 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.697308    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:08:59.839467 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:08:59.839442    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"]
Jul 24 16:08:59.841463 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:08:59.841439    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb22bacf_0695_43f1_8882_5703cd460d44.slice/crio-562abdd5c87c9c9702d57a1c9e54aa9ec0ca4bb0b2976ac0e3219342435c5365 WatchSource:0}: Error finding container 562abdd5c87c9c9702d57a1c9e54aa9ec0ca4bb0b2976ac0e3219342435c5365: Status 404 returned error can't find the container with id 562abdd5c87c9c9702d57a1c9e54aa9ec0ca4bb0b2976ac0e3219342435c5365
Jul 24 16:09:00.230434 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:00.230397    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p" event={"ID":"cb22bacf-0695-43f1-8882-5703cd460d44","Type":"ContainerStarted","Data":"f19f08e00a76b706e6439487bae358270d0af073a91a8af1688bcd9e88852b37"}
Jul 24 16:09:00.230631 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:00.230447    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p" event={"ID":"cb22bacf-0695-43f1-8882-5703cd460d44","Type":"ContainerStarted","Data":"562abdd5c87c9c9702d57a1c9e54aa9ec0ca4bb0b2976ac0e3219342435c5365"}
Jul 24 16:09:00.247932 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:00.247876    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p" podStartSLOduration=1.247857804 podStartE2EDuration="1.247857804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:08:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:09:00.246680486 +0000 UTC m=+1419.531008095" watchObservedRunningTime="2026-07-24 16:09:00.247857804 +0000 UTC m=+1419.532185410"
Jul 24 16:09:09.697664 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:09.697628    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:09:09.698086 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:09.697683    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:09:09.710208 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:09.710182    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:09:10.288568 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:10.288540    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:09:24.185766 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:24.185681    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"]
Jul 24 16:09:25.277873 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:25.277835    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"]
Jul 24 16:09:39.205329 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.205284    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq" podUID="ffa485bf-0cac-447b-bda4-7c71d76c72df" containerName="main" containerID="cri-o://271e0ac6a245d42ceaa6161678d9a460f77def903d33faeae1ad94497e9d027e" gracePeriod=45
Jul 24 16:09:39.410134 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.410101    2572 generic.go:356] "Generic (PLEG): container finished" podID="ffa485bf-0cac-447b-bda4-7c71d76c72df" containerID="271e0ac6a245d42ceaa6161678d9a460f77def903d33faeae1ad94497e9d027e" exitCode=0
Jul 24 16:09:39.410329 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.410161    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq" event={"ID":"ffa485bf-0cac-447b-bda4-7c71d76c72df","Type":"ContainerDied","Data":"271e0ac6a245d42ceaa6161678d9a460f77def903d33faeae1ad94497e9d027e"}
Jul 24 16:09:39.458931 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.458851    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:09:39.510341 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.510296    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87zgk\" (UniqueName: \"kubernetes.io/projected/ffa485bf-0cac-447b-bda4-7c71d76c72df-kube-api-access-87zgk\") pod \"ffa485bf-0cac-447b-bda4-7c71d76c72df\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") "
Jul 24 16:09:39.510341 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.510346    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-tmp-dir\") pod \"ffa485bf-0cac-447b-bda4-7c71d76c72df\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") "
Jul 24 16:09:39.510614 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.510424    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-dshm\") pod \"ffa485bf-0cac-447b-bda4-7c71d76c72df\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") "
Jul 24 16:09:39.510614 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.510450    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-home\") pod \"ffa485bf-0cac-447b-bda4-7c71d76c72df\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") "
Jul 24 16:09:39.510614 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.510478    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-model-cache\") pod \"ffa485bf-0cac-447b-bda4-7c71d76c72df\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") "
Jul 24 16:09:39.510614 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.510511    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffa485bf-0cac-447b-bda4-7c71d76c72df-tls-certs\") pod \"ffa485bf-0cac-447b-bda4-7c71d76c72df\" (UID: \"ffa485bf-0cac-447b-bda4-7c71d76c72df\") "
Jul 24 16:09:39.510788 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.510709    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ffa485bf-0cac-447b-bda4-7c71d76c72df" (UID: "ffa485bf-0cac-447b-bda4-7c71d76c72df"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:09:39.510788 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.510695    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-home" (OuterVolumeSpecName: "home") pod "ffa485bf-0cac-447b-bda4-7c71d76c72df" (UID: "ffa485bf-0cac-447b-bda4-7c71d76c72df"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:09:39.510788 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.510753    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-model-cache" (OuterVolumeSpecName: "model-cache") pod "ffa485bf-0cac-447b-bda4-7c71d76c72df" (UID: "ffa485bf-0cac-447b-bda4-7c71d76c72df"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:09:39.510970 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.510830    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-home\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:09:39.510970 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.510845    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-model-cache\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:09:39.510970 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.510857    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-tmp-dir\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:09:39.512935 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.512887    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffa485bf-0cac-447b-bda4-7c71d76c72df-kube-api-access-87zgk" (OuterVolumeSpecName: "kube-api-access-87zgk") pod "ffa485bf-0cac-447b-bda4-7c71d76c72df" (UID: "ffa485bf-0cac-447b-bda4-7c71d76c72df"). InnerVolumeSpecName "kube-api-access-87zgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:09:39.512935 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.512903    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-dshm" (OuterVolumeSpecName: "dshm") pod "ffa485bf-0cac-447b-bda4-7c71d76c72df" (UID: "ffa485bf-0cac-447b-bda4-7c71d76c72df"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:09:39.512935 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.512887    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ffa485bf-0cac-447b-bda4-7c71d76c72df-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ffa485bf-0cac-447b-bda4-7c71d76c72df" (UID: "ffa485bf-0cac-447b-bda4-7c71d76c72df"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:09:39.611948 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.611882    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffa485bf-0cac-447b-bda4-7c71d76c72df-tls-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:09:39.611948 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.611944    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87zgk\" (UniqueName: \"kubernetes.io/projected/ffa485bf-0cac-447b-bda4-7c71d76c72df-kube-api-access-87zgk\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:09:39.611948 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:39.611958    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffa485bf-0cac-447b-bda4-7c71d76c72df-dshm\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:09:40.297294 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.297219    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p" podUID="cb22bacf-0695-43f1-8882-5703cd460d44" containerName="main" containerID="cri-o://f19f08e00a76b706e6439487bae358270d0af073a91a8af1688bcd9e88852b37" gracePeriod=45
Jul 24 16:09:40.417409 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.417375    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"
Jul 24 16:09:40.417563 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.417404    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq" event={"ID":"ffa485bf-0cac-447b-bda4-7c71d76c72df","Type":"ContainerDied","Data":"2e8cfa06feddada4d71e9d3f1791f3af4536f32cd58e869cd0d0288c6b5592a1"}
Jul 24 16:09:40.417563 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.417471    2572 scope.go:117] "RemoveContainer" containerID="271e0ac6a245d42ceaa6161678d9a460f77def903d33faeae1ad94497e9d027e"
Jul 24 16:09:40.419371 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.419339    2572 generic.go:356] "Generic (PLEG): container finished" podID="cb22bacf-0695-43f1-8882-5703cd460d44" containerID="f19f08e00a76b706e6439487bae358270d0af073a91a8af1688bcd9e88852b37" exitCode=0
Jul 24 16:09:40.419512 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.419410    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p" event={"ID":"cb22bacf-0695-43f1-8882-5703cd460d44","Type":"ContainerDied","Data":"f19f08e00a76b706e6439487bae358270d0af073a91a8af1688bcd9e88852b37"}
Jul 24 16:09:40.455811 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.455771    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"]
Jul 24 16:09:40.457903 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.457872    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-0d2b906a-kserve-6dd7bf94c5-6hmlq"]
Jul 24 16:09:40.560972 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.560866    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:09:40.620687 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.620648    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7j94q\" (UniqueName: \"kubernetes.io/projected/cb22bacf-0695-43f1-8882-5703cd460d44-kube-api-access-7j94q\") pod \"cb22bacf-0695-43f1-8882-5703cd460d44\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") "
Jul 24 16:09:40.620687 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.620701    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-dshm\") pod \"cb22bacf-0695-43f1-8882-5703cd460d44\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") "
Jul 24 16:09:40.621035 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.620732    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-tmp-dir\") pod \"cb22bacf-0695-43f1-8882-5703cd460d44\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") "
Jul 24 16:09:40.621035 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.620752    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-home\") pod \"cb22bacf-0695-43f1-8882-5703cd460d44\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") "
Jul 24 16:09:40.621035 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.620788    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-model-cache\") pod \"cb22bacf-0695-43f1-8882-5703cd460d44\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") "
Jul 24 16:09:40.621035 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.620845    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb22bacf-0695-43f1-8882-5703cd460d44-tls-certs\") pod \"cb22bacf-0695-43f1-8882-5703cd460d44\" (UID: \"cb22bacf-0695-43f1-8882-5703cd460d44\") "
Jul 24 16:09:40.621253 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.621043    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-home" (OuterVolumeSpecName: "home") pod "cb22bacf-0695-43f1-8882-5703cd460d44" (UID: "cb22bacf-0695-43f1-8882-5703cd460d44"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:09:40.621253 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.621071    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cb22bacf-0695-43f1-8882-5703cd460d44" (UID: "cb22bacf-0695-43f1-8882-5703cd460d44"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:09:40.621253 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.621089    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-model-cache" (OuterVolumeSpecName: "model-cache") pod "cb22bacf-0695-43f1-8882-5703cd460d44" (UID: "cb22bacf-0695-43f1-8882-5703cd460d44"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:09:40.621253 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.621161    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-tmp-dir\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:09:40.621253 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.621173    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-home\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:09:40.621253 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.621181    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-model-cache\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:09:40.623121 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.623093    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-dshm" (OuterVolumeSpecName: "dshm") pod "cb22bacf-0695-43f1-8882-5703cd460d44" (UID: "cb22bacf-0695-43f1-8882-5703cd460d44"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:09:40.623229 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.623156    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb22bacf-0695-43f1-8882-5703cd460d44-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cb22bacf-0695-43f1-8882-5703cd460d44" (UID: "cb22bacf-0695-43f1-8882-5703cd460d44"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:09:40.623229 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.623184    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb22bacf-0695-43f1-8882-5703cd460d44-kube-api-access-7j94q" (OuterVolumeSpecName: "kube-api-access-7j94q") pod "cb22bacf-0695-43f1-8882-5703cd460d44" (UID: "cb22bacf-0695-43f1-8882-5703cd460d44"). InnerVolumeSpecName "kube-api-access-7j94q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:09:40.722610 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.722568    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb22bacf-0695-43f1-8882-5703cd460d44-tls-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:09:40.722610 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.722601    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j94q\" (UniqueName: \"kubernetes.io/projected/cb22bacf-0695-43f1-8882-5703cd460d44-kube-api-access-7j94q\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:09:40.722610 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:40.722610    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb22bacf-0695-43f1-8882-5703cd460d44-dshm\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:09:41.222836 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:41.222798    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffa485bf-0cac-447b-bda4-7c71d76c72df" path="/var/lib/kubelet/pods/ffa485bf-0cac-447b-bda4-7c71d76c72df/volumes"
Jul 24 16:09:41.425103 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:41.425076    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"
Jul 24 16:09:41.425558 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:41.425118    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p" event={"ID":"cb22bacf-0695-43f1-8882-5703cd460d44","Type":"ContainerDied","Data":"562abdd5c87c9c9702d57a1c9e54aa9ec0ca4bb0b2976ac0e3219342435c5365"}
Jul 24 16:09:41.425558 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:41.425176    2572 scope.go:117] "RemoveContainer" containerID="f19f08e00a76b706e6439487bae358270d0af073a91a8af1688bcd9e88852b37"
Jul 24 16:09:41.443516 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:41.443489    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"]
Jul 24 16:09:41.449312 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:41.449287    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-0d2b906a-kserve-5769dd9f-z6l9p"]
Jul 24 16:09:43.222667 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:09:43.222632    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb22bacf-0695-43f1-8882-5703cd460d44" path="/var/lib/kubelet/pods/cb22bacf-0695-43f1-8882-5703cd460d44/volumes"
Jul 24 16:10:21.195806 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:10:21.195774    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 16:10:21.202812 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:10:21.202789    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 16:13:13.447798 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.447758    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"]
Jul 24 16:13:13.450298 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.448218    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffa485bf-0cac-447b-bda4-7c71d76c72df" containerName="main"
Jul 24 16:13:13.450298 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.448232    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffa485bf-0cac-447b-bda4-7c71d76c72df" containerName="main"
Jul 24 16:13:13.450298 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.448248    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb22bacf-0695-43f1-8882-5703cd460d44" containerName="main"
Jul 24 16:13:13.450298 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.448257    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb22bacf-0695-43f1-8882-5703cd460d44" containerName="main"
Jul 24 16:13:13.450298 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.448337    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffa485bf-0cac-447b-bda4-7c71d76c72df" containerName="main"
Jul 24 16:13:13.450298 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.448348    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb22bacf-0695-43f1-8882-5703cd460d44" containerName="main"
Jul 24 16:13:13.451233 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.451217    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.453779 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.453754    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c3420287-openshift-default-dockercfg-dcwrd\""
Jul 24 16:13:13.475799 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.475767    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"]
Jul 24 16:13:13.582044 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.582006    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-envoy\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.582044 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.582042    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-podinfo\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.582260 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.582091    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-workload-socket\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.582260 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.582115    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-workload-certs\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.582260 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.582140    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e523c92e-73a8-4ef0-ab0a-2accf416750f-istiod-ca-cert\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.582260 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.582218    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cqsb\" (UniqueName: \"kubernetes.io/projected/e523c92e-73a8-4ef0-ab0a-2accf416750f-kube-api-access-8cqsb\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.582403 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.582262    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-credential-socket\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.582403 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.582305    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-token\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.582403 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.582335    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-data\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.683023 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.682984    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e523c92e-73a8-4ef0-ab0a-2accf416750f-istiod-ca-cert\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.683023 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.683026    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cqsb\" (UniqueName: \"kubernetes.io/projected/e523c92e-73a8-4ef0-ab0a-2accf416750f-kube-api-access-8cqsb\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.683239 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.683044    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-credential-socket\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.683239 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.683164    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-token\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.683239 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.683206    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-data\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.683383 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.683277    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-envoy\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.683383 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.683304    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-podinfo\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.683383 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.683376    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-workload-socket\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.683554 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.683417    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-credential-socket\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.683554 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.683419    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-workload-certs\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.683669 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.683626    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-data\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.683788 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.683769    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e523c92e-73a8-4ef0-ab0a-2accf416750f-istiod-ca-cert\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.683854 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.683789    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-workload-certs\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.683854 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.683768    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-workload-socket\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.685812 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.685791    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-envoy\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.686038 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.686010    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-podinfo\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.691356 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.691328    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-token\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.692393 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.692366    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cqsb\" (UniqueName: \"kubernetes.io/projected/e523c92e-73a8-4ef0-ab0a-2accf416750f-kube-api-access-8cqsb\") pod \"e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") " pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.762079 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.761989    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:13.922441 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.922413    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"]
Jul 24 16:13:13.924965 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:13:13.924934    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode523c92e_73a8_4ef0_ab0a_2accf416750f.slice/crio-ecd592005e4738e48b9878538bfda6e58518cdb33b260496151623c8a1f8510f WatchSource:0}: Error finding container ecd592005e4738e48b9878538bfda6e58518cdb33b260496151623c8a1f8510f: Status 404 returned error can't find the container with id ecd592005e4738e48b9878538bfda6e58518cdb33b260496151623c8a1f8510f
Jul 24 16:13:13.927019 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.926984    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:13:13.927104 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.927057    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:13:13.927104 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:13.927088    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:13:14.345036 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:14.345002    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp" event={"ID":"e523c92e-73a8-4ef0-ab0a-2accf416750f","Type":"ContainerStarted","Data":"2b616f227ff373dd6ccc8802d8ef472cdb0ce49059fb6860d63211f791122cc1"}
Jul 24 16:13:14.345230 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:14.345057    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp" event={"ID":"e523c92e-73a8-4ef0-ab0a-2accf416750f","Type":"ContainerStarted","Data":"ecd592005e4738e48b9878538bfda6e58518cdb33b260496151623c8a1f8510f"}
Jul 24 16:13:14.368977 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:14.368906    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp" podStartSLOduration=1.3688905359999999 podStartE2EDuration="1.368890536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:13:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:13:14.368262137 +0000 UTC m=+1673.652589772" watchObservedRunningTime="2026-07-24 16:13:14.368890536 +0000 UTC m=+1673.653218144"
Jul 24 16:13:14.762883 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:14.762848    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:14.767869 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:14.767839    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:15.349084 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:15.349052    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:15.350023 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:15.349999    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:27.404009 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:27.403974    2572 generic.go:356] "Generic (PLEG): container finished" podID="e523c92e-73a8-4ef0-ab0a-2accf416750f" containerID="2b616f227ff373dd6ccc8802d8ef472cdb0ce49059fb6860d63211f791122cc1" exitCode=0
Jul 24 16:13:27.404442 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:27.404056    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp" event={"ID":"e523c92e-73a8-4ef0-ab0a-2accf416750f","Type":"ContainerDied","Data":"2b616f227ff373dd6ccc8802d8ef472cdb0ce49059fb6860d63211f791122cc1"}
Jul 24 16:13:27.404486 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:27.404468    2572 scope.go:117] "RemoveContainer" containerID="2b616f227ff373dd6ccc8802d8ef472cdb0ce49059fb6860d63211f791122cc1"
Jul 24 16:13:27.405514 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:27.405485    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:13:27.405585 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:27.405570    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:13:27.405631 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:27.405616    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:13:28.409943 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:28.409887    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp" event={"ID":"e523c92e-73a8-4ef0-ab0a-2accf416750f","Type":"ContainerStarted","Data":"1e48e2d66ac4197b9e3b069778be9736361a5105842c9685c91c9c4b36a62791"}
Jul 24 16:13:28.763216 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:28.763104    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:28.764401 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:28.764370    2572 patch_prober.go:28] interesting pod/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Jul 24 16:13:28.764527 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:28.764427    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp" podUID="e523c92e-73a8-4ef0-ab0a-2accf416750f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Jul 24 16:13:29.762635 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:29.762598    2572 patch_prober.go:28] interesting pod/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Jul 24 16:13:29.763051 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:29.762653    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp" podUID="e523c92e-73a8-4ef0-ab0a-2accf416750f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Jul 24 16:13:30.349988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:30.349947    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:30.762437 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:30.762397    2572 patch_prober.go:28] interesting pod/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Jul 24 16:13:30.762634 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:30.762466    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp" podUID="e523c92e-73a8-4ef0-ab0a-2accf416750f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Jul 24 16:13:31.762527 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:31.762494    2572 patch_prober.go:28] interesting pod/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Jul 24 16:13:31.762891 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:31.762552    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp" podUID="e523c92e-73a8-4ef0-ab0a-2accf416750f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Jul 24 16:13:32.763405 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:32.763364    2572 patch_prober.go:28] interesting pod/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Jul 24 16:13:32.763808 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:32.763441    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp" podUID="e523c92e-73a8-4ef0-ab0a-2accf416750f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Jul 24 16:13:33.766953 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:33.766901    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:33.767856 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:33.767838    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:13:56.586233 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:56.586013    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"]
Jul 24 16:13:56.587594 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:56.587555    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp" podUID="e523c92e-73a8-4ef0-ab0a-2accf416750f" containerName="istio-proxy" containerID="cri-o://1e48e2d66ac4197b9e3b069778be9736361a5105842c9685c91c9c4b36a62791" gracePeriod=30
Jul 24 16:13:57.625564 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.625532    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"]
Jul 24 16:13:57.628836 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.628818    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.631425 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.631407    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-c7a36573-gw-openshift-default-dockercfg-5ch85\""
Jul 24 16:13:57.646494 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.646471    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"]
Jul 24 16:13:57.700299 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.700270    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-data\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.700491 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.700309    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-podinfo\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.700491 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.700330    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-credential-socket\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.700491 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.700370    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-envoy\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.700491 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.700399    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mgj5\" (UniqueName: \"kubernetes.io/projected/da2062ff-4747-4248-a0b6-9ad43f94c11e-kube-api-access-5mgj5\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.700491 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.700428    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-workload-socket\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.700491 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.700469    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da2062ff-4747-4248-a0b6-9ad43f94c11e-istiod-ca-cert\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.700791 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.700497    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-workload-certs\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.700791 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.700541    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-token\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.801893 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.801852    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-data\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.801893 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.801896    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-podinfo\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.802167 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.801937    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-credential-socket\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.802167 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.801960    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-envoy\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.802167 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.801986    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mgj5\" (UniqueName: \"kubernetes.io/projected/da2062ff-4747-4248-a0b6-9ad43f94c11e-kube-api-access-5mgj5\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.802167 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.802024    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-workload-socket\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.802167 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.802062    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da2062ff-4747-4248-a0b6-9ad43f94c11e-istiod-ca-cert\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.802167 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.802101    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-workload-certs\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.802469 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.802352    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-data\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.802469 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.802404    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-token\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.802586 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.802462    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-workload-socket\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.802663 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.802641    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-workload-certs\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.802885 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.802855    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-credential-socket\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.803028 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.802898    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da2062ff-4747-4248-a0b6-9ad43f94c11e-istiod-ca-cert\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.804998 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.804976    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-envoy\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.805318 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.805297    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-podinfo\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.814566 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.814538    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-token\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.814791 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.814772    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mgj5\" (UniqueName: \"kubernetes.io/projected/da2062ff-4747-4248-a0b6-9ad43f94c11e-kube-api-access-5mgj5\") pod \"e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") " pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:57.939122 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:57.939085    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:58.100339 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:58.100312    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"]
Jul 24 16:13:58.102890 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:13:58.102865    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda2062ff_4747_4248_a0b6_9ad43f94c11e.slice/crio-9a9ac7cdf8ce6ccab449a04caed215c7fc6cce4bbbfa50c23a6dca7b9bf60b68 WatchSource:0}: Error finding container 9a9ac7cdf8ce6ccab449a04caed215c7fc6cce4bbbfa50c23a6dca7b9bf60b68: Status 404 returned error can't find the container with id 9a9ac7cdf8ce6ccab449a04caed215c7fc6cce4bbbfa50c23a6dca7b9bf60b68
Jul 24 16:13:58.104912 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:58.104879    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:13:58.105041 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:58.104970    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:13:58.105041 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:58.105002    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:13:58.544340 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:58.544306    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498" event={"ID":"da2062ff-4747-4248-a0b6-9ad43f94c11e","Type":"ContainerStarted","Data":"e1cdaa9fd85dca9a9d17580534185c47218793153b50eec300109c39c29d8146"}
Jul 24 16:13:58.544522 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:58.544356    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498" event={"ID":"da2062ff-4747-4248-a0b6-9ad43f94c11e","Type":"ContainerStarted","Data":"9a9ac7cdf8ce6ccab449a04caed215c7fc6cce4bbbfa50c23a6dca7b9bf60b68"}
Jul 24 16:13:58.583032 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:58.582974    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498" podStartSLOduration=1.582953684 podStartE2EDuration="1.582953684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:13:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:13:58.581645241 +0000 UTC m=+1717.865972869" watchObservedRunningTime="2026-07-24 16:13:58.582953684 +0000 UTC m=+1717.867281290"
Jul 24 16:13:58.940171 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:58.940136    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:13:59.945415 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:13:59.945381    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:14:00.552420 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:00.552379    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:14:00.553500 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:00.553466    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:14:01.838750 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.838723    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:14:01.942149 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.942113    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e523c92e-73a8-4ef0-ab0a-2accf416750f-istiod-ca-cert\") pod \"e523c92e-73a8-4ef0-ab0a-2accf416750f\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") "
Jul 24 16:14:01.942149 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.942148    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cqsb\" (UniqueName: \"kubernetes.io/projected/e523c92e-73a8-4ef0-ab0a-2accf416750f-kube-api-access-8cqsb\") pod \"e523c92e-73a8-4ef0-ab0a-2accf416750f\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") "
Jul 24 16:14:01.942361 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.942182    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-data\") pod \"e523c92e-73a8-4ef0-ab0a-2accf416750f\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") "
Jul 24 16:14:01.942361 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.942212    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-workload-certs\") pod \"e523c92e-73a8-4ef0-ab0a-2accf416750f\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") "
Jul 24 16:14:01.942361 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.942326    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-token\") pod \"e523c92e-73a8-4ef0-ab0a-2accf416750f\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") "
Jul 24 16:14:01.942513 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.942431    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-podinfo\") pod \"e523c92e-73a8-4ef0-ab0a-2accf416750f\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") "
Jul 24 16:14:01.942513 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.942458    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-workload-socket\") pod \"e523c92e-73a8-4ef0-ab0a-2accf416750f\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") "
Jul 24 16:14:01.942513 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.942485    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-envoy\") pod \"e523c92e-73a8-4ef0-ab0a-2accf416750f\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") "
Jul 24 16:14:01.942658 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.942519    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-credential-socket\") pod \"e523c92e-73a8-4ef0-ab0a-2accf416750f\" (UID: \"e523c92e-73a8-4ef0-ab0a-2accf416750f\") "
Jul 24 16:14:01.942658 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.942525    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e523c92e-73a8-4ef0-ab0a-2accf416750f" (UID: "e523c92e-73a8-4ef0-ab0a-2accf416750f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:14:01.942658 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.942591    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e523c92e-73a8-4ef0-ab0a-2accf416750f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e523c92e-73a8-4ef0-ab0a-2accf416750f" (UID: "e523c92e-73a8-4ef0-ab0a-2accf416750f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:14:01.942813 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.942736    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e523c92e-73a8-4ef0-ab0a-2accf416750f" (UID: "e523c92e-73a8-4ef0-ab0a-2accf416750f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:14:01.942870 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.942822    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-workload-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:14:01.942870 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.942840    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-workload-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:14:01.942870 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.942855    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e523c92e-73a8-4ef0-ab0a-2accf416750f-istiod-ca-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:14:01.943043 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.942937    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e523c92e-73a8-4ef0-ab0a-2accf416750f" (UID: "e523c92e-73a8-4ef0-ab0a-2accf416750f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:14:01.943472 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.943439    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-data" (OuterVolumeSpecName: "istio-data") pod "e523c92e-73a8-4ef0-ab0a-2accf416750f" (UID: "e523c92e-73a8-4ef0-ab0a-2accf416750f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:14:01.944954 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.944904    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-token" (OuterVolumeSpecName: "istio-token") pod "e523c92e-73a8-4ef0-ab0a-2accf416750f" (UID: "e523c92e-73a8-4ef0-ab0a-2accf416750f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:14:01.944954 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.944946    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e523c92e-73a8-4ef0-ab0a-2accf416750f" (UID: "e523c92e-73a8-4ef0-ab0a-2accf416750f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:14:01.945098 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.944947    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e523c92e-73a8-4ef0-ab0a-2accf416750f" (UID: "e523c92e-73a8-4ef0-ab0a-2accf416750f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 16:14:01.945098 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:01.944962    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e523c92e-73a8-4ef0-ab0a-2accf416750f-kube-api-access-8cqsb" (OuterVolumeSpecName: "kube-api-access-8cqsb") pod "e523c92e-73a8-4ef0-ab0a-2accf416750f" (UID: "e523c92e-73a8-4ef0-ab0a-2accf416750f"). InnerVolumeSpecName "kube-api-access-8cqsb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:14:02.043698 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:02.043647    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-credential-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:14:02.043698 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:02.043692    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cqsb\" (UniqueName: \"kubernetes.io/projected/e523c92e-73a8-4ef0-ab0a-2accf416750f-kube-api-access-8cqsb\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:14:02.043698 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:02.043703    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-data\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:14:02.043698 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:02.043713    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-token\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:14:02.043698 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:02.043721    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-podinfo\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:14:02.044042 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:02.043728    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e523c92e-73a8-4ef0-ab0a-2accf416750f-istio-envoy\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:14:02.563871 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:02.563828    2572 generic.go:356] "Generic (PLEG): container finished" podID="e523c92e-73a8-4ef0-ab0a-2accf416750f" containerID="1e48e2d66ac4197b9e3b069778be9736361a5105842c9685c91c9c4b36a62791" exitCode=0
Jul 24 16:14:02.564126 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:02.563879    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"
Jul 24 16:14:02.564126 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:02.563904    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp" event={"ID":"e523c92e-73a8-4ef0-ab0a-2accf416750f","Type":"ContainerDied","Data":"1e48e2d66ac4197b9e3b069778be9736361a5105842c9685c91c9c4b36a62791"}
Jul 24 16:14:02.564126 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:02.563981    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp" event={"ID":"e523c92e-73a8-4ef0-ab0a-2accf416750f","Type":"ContainerDied","Data":"ecd592005e4738e48b9878538bfda6e58518cdb33b260496151623c8a1f8510f"}
Jul 24 16:14:02.564126 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:02.564004    2572 scope.go:117] "RemoveContainer" containerID="1e48e2d66ac4197b9e3b069778be9736361a5105842c9685c91c9c4b36a62791"
Jul 24 16:14:02.573034 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:02.573016    2572 scope.go:117] "RemoveContainer" containerID="2b616f227ff373dd6ccc8802d8ef472cdb0ce49059fb6860d63211f791122cc1"
Jul 24 16:14:02.580994 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:02.580976    2572 scope.go:117] "RemoveContainer" containerID="1e48e2d66ac4197b9e3b069778be9736361a5105842c9685c91c9c4b36a62791"
Jul 24 16:14:02.581249 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:14:02.581231    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e48e2d66ac4197b9e3b069778be9736361a5105842c9685c91c9c4b36a62791\": container with ID starting with 1e48e2d66ac4197b9e3b069778be9736361a5105842c9685c91c9c4b36a62791 not found: ID does not exist" containerID="1e48e2d66ac4197b9e3b069778be9736361a5105842c9685c91c9c4b36a62791"
Jul 24 16:14:02.581298 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:02.581259    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e48e2d66ac4197b9e3b069778be9736361a5105842c9685c91c9c4b36a62791"} err="failed to get container status \"1e48e2d66ac4197b9e3b069778be9736361a5105842c9685c91c9c4b36a62791\": rpc error: code = NotFound desc = could not find container \"1e48e2d66ac4197b9e3b069778be9736361a5105842c9685c91c9c4b36a62791\": container with ID starting with 1e48e2d66ac4197b9e3b069778be9736361a5105842c9685c91c9c4b36a62791 not found: ID does not exist"
Jul 24 16:14:02.581298 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:02.581275    2572 scope.go:117] "RemoveContainer" containerID="2b616f227ff373dd6ccc8802d8ef472cdb0ce49059fb6860d63211f791122cc1"
Jul 24 16:14:02.581485 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:14:02.581470    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b616f227ff373dd6ccc8802d8ef472cdb0ce49059fb6860d63211f791122cc1\": container with ID starting with 2b616f227ff373dd6ccc8802d8ef472cdb0ce49059fb6860d63211f791122cc1 not found: ID does not exist" containerID="2b616f227ff373dd6ccc8802d8ef472cdb0ce49059fb6860d63211f791122cc1"
Jul 24 16:14:02.581530 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:02.581489    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b616f227ff373dd6ccc8802d8ef472cdb0ce49059fb6860d63211f791122cc1"} err="failed to get container status \"2b616f227ff373dd6ccc8802d8ef472cdb0ce49059fb6860d63211f791122cc1\": rpc error: code = NotFound desc = could not find container \"2b616f227ff373dd6ccc8802d8ef472cdb0ce49059fb6860d63211f791122cc1\": container with ID starting with 2b616f227ff373dd6ccc8802d8ef472cdb0ce49059fb6860d63211f791122cc1 not found: ID does not exist"
Jul 24 16:14:02.587758 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:02.587727    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"]
Jul 24 16:14:02.593996 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:02.593965    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c3420287-openshift-default-65c7dcf54-xlprp"]
Jul 24 16:14:03.223299 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:03.223267    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e523c92e-73a8-4ef0-ab0a-2accf416750f" path="/var/lib/kubelet/pods/e523c92e-73a8-4ef0-ab0a-2accf416750f/volumes"
Jul 24 16:14:15.554638 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:15.554595    2572 patch_prober.go:28] interesting pod/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Jul 24 16:14:15.555080 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:15.554668    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498" podUID="da2062ff-4747-4248-a0b6-9ad43f94c11e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Jul 24 16:14:15.627100 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:15.627066    2572 generic.go:356] "Generic (PLEG): container finished" podID="da2062ff-4747-4248-a0b6-9ad43f94c11e" containerID="e1cdaa9fd85dca9a9d17580534185c47218793153b50eec300109c39c29d8146" exitCode=0
Jul 24 16:14:15.627280 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:15.627139    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498" event={"ID":"da2062ff-4747-4248-a0b6-9ad43f94c11e","Type":"ContainerDied","Data":"e1cdaa9fd85dca9a9d17580534185c47218793153b50eec300109c39c29d8146"}
Jul 24 16:14:15.627565 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:15.627550    2572 scope.go:117] "RemoveContainer" containerID="e1cdaa9fd85dca9a9d17580534185c47218793153b50eec300109c39c29d8146"
Jul 24 16:14:15.628642 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:15.628611    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:14:15.628719 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:15.628702    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:14:15.628773 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:15.628760    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:14:16.632980 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:16.632947    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498" event={"ID":"da2062ff-4747-4248-a0b6-9ad43f94c11e","Type":"ContainerStarted","Data":"ab36eb187e9b9be70a530b2e661da72ca4d42129d476135defc1139cd0a8b993"}
Jul 24 16:14:16.939809 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:16.939771    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:14:16.940089 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:16.940067    2572 patch_prober.go:28] interesting pod/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Jul 24 16:14:16.940185 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:16.940137    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498" podUID="da2062ff-4747-4248-a0b6-9ad43f94c11e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Jul 24 16:14:17.939779 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:17.939738    2572 patch_prober.go:28] interesting pod/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Jul 24 16:14:17.940229 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:17.939797    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498" podUID="da2062ff-4747-4248-a0b6-9ad43f94c11e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Jul 24 16:14:18.940091 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:18.940050    2572 patch_prober.go:28] interesting pod/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Jul 24 16:14:18.940512 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:18.940109    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498" podUID="da2062ff-4747-4248-a0b6-9ad43f94c11e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Jul 24 16:14:19.940194 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:19.940160    2572 patch_prober.go:28] interesting pod/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Jul 24 16:14:19.940619 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:19.940219    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498" podUID="da2062ff-4747-4248-a0b6-9ad43f94c11e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Jul 24 16:14:20.944061 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:20.944031    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:14:20.944449 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:20.944268    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:14:20.945093 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:20.945076    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:14:40.283441 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:40.283403    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"]
Jul 24 16:14:40.284015 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:40.283653    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498" podUID="da2062ff-4747-4248-a0b6-9ad43f94c11e" containerName="istio-proxy" containerID="cri-o://ab36eb187e9b9be70a530b2e661da72ca4d42129d476135defc1139cd0a8b993" gracePeriod=30
Jul 24 16:14:41.086481 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.086445    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"]
Jul 24 16:14:41.086797 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.086779    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e523c92e-73a8-4ef0-ab0a-2accf416750f" containerName="istio-proxy"
Jul 24 16:14:41.086797 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.086795    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e523c92e-73a8-4ef0-ab0a-2accf416750f" containerName="istio-proxy"
Jul 24 16:14:41.086982 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.086856    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e523c92e-73a8-4ef0-ab0a-2accf416750f" containerName="istio-proxy"
Jul 24 16:14:41.091171 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.091148    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.094133 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.094109    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-518181e2-openshift-default-dockercfg-99xf6\""
Jul 24 16:14:41.104938 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.104897    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"]
Jul 24 16:14:41.208145 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.208106    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-podinfo\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.208342 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.208155    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-workload-certs\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.208342 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.208193    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e85f31e-ae19-4373-9282-fdddb3ac273e-istiod-ca-cert\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.208342 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.208255    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-workload-socket\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.208342 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.208304    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-data\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.208342 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.208324    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9pqp\" (UniqueName: \"kubernetes.io/projected/1e85f31e-ae19-4373-9282-fdddb3ac273e-kube-api-access-c9pqp\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.208342 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.208344    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-token\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.208656 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.208415    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-credential-socket\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.208656 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.208500    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-envoy\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.309029 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.308990    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-data\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.309029 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.309030    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9pqp\" (UniqueName: \"kubernetes.io/projected/1e85f31e-ae19-4373-9282-fdddb3ac273e-kube-api-access-c9pqp\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.309581 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.309051    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-token\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.309581 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.309080    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-credential-socket\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.309581 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.309139    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-envoy\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.309581 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.309181    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-podinfo\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.309581 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.309211    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-workload-certs\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.309581 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.309253    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e85f31e-ae19-4373-9282-fdddb3ac273e-istiod-ca-cert\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.309581 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.309291    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-workload-socket\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.309581 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.309444    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-data\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.309581 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.309499    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-credential-socket\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.310112 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.310089    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e85f31e-ae19-4373-9282-fdddb3ac273e-istiod-ca-cert\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.310292 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.310271    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-workload-socket\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.310330 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.310307    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-workload-certs\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.311800 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.311776    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-envoy\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.311909 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.311892    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-podinfo\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.319891 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.319854    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9pqp\" (UniqueName: \"kubernetes.io/projected/1e85f31e-ae19-4373-9282-fdddb3ac273e-kube-api-access-c9pqp\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.320080 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.319955    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-token\") pod \"e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") " pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.401346 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.401308    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:41.541808 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.541779    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"]
Jul 24 16:14:41.543997 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:14:41.543969    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e85f31e_ae19_4373_9282_fdddb3ac273e.slice/crio-4a5ca623410c95fd0ab81f0c47792632e4268955a6213543cfd8fc4f5102ddd0 WatchSource:0}: Error finding container 4a5ca623410c95fd0ab81f0c47792632e4268955a6213543cfd8fc4f5102ddd0: Status 404 returned error can't find the container with id 4a5ca623410c95fd0ab81f0c47792632e4268955a6213543cfd8fc4f5102ddd0
Jul 24 16:14:41.546232 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.546202    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:14:41.546304 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.546267    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:14:41.546304 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.546293    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:14:41.744522 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.744420    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn" event={"ID":"1e85f31e-ae19-4373-9282-fdddb3ac273e","Type":"ContainerStarted","Data":"65490cd11c1040d3741b42cd8ae388370bc25654df239461fde6eb33652ed8cf"}
Jul 24 16:14:41.744522 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.744481    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn" event={"ID":"1e85f31e-ae19-4373-9282-fdddb3ac273e","Type":"ContainerStarted","Data":"4a5ca623410c95fd0ab81f0c47792632e4268955a6213543cfd8fc4f5102ddd0"}
Jul 24 16:14:41.766810 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:41.766758    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn" podStartSLOduration=0.766741405 podStartE2EDuration="766.741405ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:14:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:14:41.764350362 +0000 UTC m=+1761.048677969" watchObservedRunningTime="2026-07-24 16:14:41.766741405 +0000 UTC m=+1761.051069069"
Jul 24 16:14:42.402121 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:42.402079    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:42.406871 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:42.406843    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:42.750100 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:42.750013    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:42.751078 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:42.751056    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:14:45.529246 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.529222    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:14:45.654573 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.654542    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-workload-certs\") pod \"da2062ff-4747-4248-a0b6-9ad43f94c11e\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") "
Jul 24 16:14:45.654765 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.654583    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-token\") pod \"da2062ff-4747-4248-a0b6-9ad43f94c11e\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") "
Jul 24 16:14:45.654765 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.654627    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da2062ff-4747-4248-a0b6-9ad43f94c11e-istiod-ca-cert\") pod \"da2062ff-4747-4248-a0b6-9ad43f94c11e\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") "
Jul 24 16:14:45.654765 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.654652    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-envoy\") pod \"da2062ff-4747-4248-a0b6-9ad43f94c11e\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") "
Jul 24 16:14:45.654765 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.654704    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-podinfo\") pod \"da2062ff-4747-4248-a0b6-9ad43f94c11e\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") "
Jul 24 16:14:45.654765 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.654726    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-workload-socket\") pod \"da2062ff-4747-4248-a0b6-9ad43f94c11e\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") "
Jul 24 16:14:45.654765 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.654757    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-credential-socket\") pod \"da2062ff-4747-4248-a0b6-9ad43f94c11e\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") "
Jul 24 16:14:45.655092 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.654784    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mgj5\" (UniqueName: \"kubernetes.io/projected/da2062ff-4747-4248-a0b6-9ad43f94c11e-kube-api-access-5mgj5\") pod \"da2062ff-4747-4248-a0b6-9ad43f94c11e\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") "
Jul 24 16:14:45.655092 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.654868    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-data\") pod \"da2062ff-4747-4248-a0b6-9ad43f94c11e\" (UID: \"da2062ff-4747-4248-a0b6-9ad43f94c11e\") "
Jul 24 16:14:45.655202 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.655081    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "da2062ff-4747-4248-a0b6-9ad43f94c11e" (UID: "da2062ff-4747-4248-a0b6-9ad43f94c11e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:14:45.655202 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.655090    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da2062ff-4747-4248-a0b6-9ad43f94c11e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "da2062ff-4747-4248-a0b6-9ad43f94c11e" (UID: "da2062ff-4747-4248-a0b6-9ad43f94c11e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:14:45.655202 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.655087    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "da2062ff-4747-4248-a0b6-9ad43f94c11e" (UID: "da2062ff-4747-4248-a0b6-9ad43f94c11e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:14:45.655354 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.655214    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/da2062ff-4747-4248-a0b6-9ad43f94c11e-istiod-ca-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:14:45.655354 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.655236    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-workload-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:14:45.655354 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.655272    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-credential-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:14:45.655354 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.655324    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "da2062ff-4747-4248-a0b6-9ad43f94c11e" (UID: "da2062ff-4747-4248-a0b6-9ad43f94c11e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:14:45.655831 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.655811    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-data" (OuterVolumeSpecName: "istio-data") pod "da2062ff-4747-4248-a0b6-9ad43f94c11e" (UID: "da2062ff-4747-4248-a0b6-9ad43f94c11e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:14:45.657182 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.657158    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-token" (OuterVolumeSpecName: "istio-token") pod "da2062ff-4747-4248-a0b6-9ad43f94c11e" (UID: "da2062ff-4747-4248-a0b6-9ad43f94c11e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:14:45.657272 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.657159    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da2062ff-4747-4248-a0b6-9ad43f94c11e-kube-api-access-5mgj5" (OuterVolumeSpecName: "kube-api-access-5mgj5") pod "da2062ff-4747-4248-a0b6-9ad43f94c11e" (UID: "da2062ff-4747-4248-a0b6-9ad43f94c11e"). InnerVolumeSpecName "kube-api-access-5mgj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:14:45.657442 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.657423    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "da2062ff-4747-4248-a0b6-9ad43f94c11e" (UID: "da2062ff-4747-4248-a0b6-9ad43f94c11e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 16:14:45.657513 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.657437    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "da2062ff-4747-4248-a0b6-9ad43f94c11e" (UID: "da2062ff-4747-4248-a0b6-9ad43f94c11e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:14:45.756303 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.756267    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-data\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:14:45.756303 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.756297    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-workload-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:14:45.756303 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.756306    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-token\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:14:45.756303 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.756313    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-envoy\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:14:45.756556 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.756321    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/da2062ff-4747-4248-a0b6-9ad43f94c11e-istio-podinfo\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:14:45.756556 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.756330    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mgj5\" (UniqueName: \"kubernetes.io/projected/da2062ff-4747-4248-a0b6-9ad43f94c11e-kube-api-access-5mgj5\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:14:45.766062 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.766031    2572 generic.go:356] "Generic (PLEG): container finished" podID="da2062ff-4747-4248-a0b6-9ad43f94c11e" containerID="ab36eb187e9b9be70a530b2e661da72ca4d42129d476135defc1139cd0a8b993" exitCode=0
Jul 24 16:14:45.766219 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.766082    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"
Jul 24 16:14:45.766219 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.766125    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498" event={"ID":"da2062ff-4747-4248-a0b6-9ad43f94c11e","Type":"ContainerDied","Data":"ab36eb187e9b9be70a530b2e661da72ca4d42129d476135defc1139cd0a8b993"}
Jul 24 16:14:45.766219 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.766180    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498" event={"ID":"da2062ff-4747-4248-a0b6-9ad43f94c11e","Type":"ContainerDied","Data":"9a9ac7cdf8ce6ccab449a04caed215c7fc6cce4bbbfa50c23a6dca7b9bf60b68"}
Jul 24 16:14:45.766219 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.766209    2572 scope.go:117] "RemoveContainer" containerID="ab36eb187e9b9be70a530b2e661da72ca4d42129d476135defc1139cd0a8b993"
Jul 24 16:14:45.775274 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.775254    2572 scope.go:117] "RemoveContainer" containerID="e1cdaa9fd85dca9a9d17580534185c47218793153b50eec300109c39c29d8146"
Jul 24 16:14:45.782740 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.782717    2572 scope.go:117] "RemoveContainer" containerID="ab36eb187e9b9be70a530b2e661da72ca4d42129d476135defc1139cd0a8b993"
Jul 24 16:14:45.782981 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:14:45.782959    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab36eb187e9b9be70a530b2e661da72ca4d42129d476135defc1139cd0a8b993\": container with ID starting with ab36eb187e9b9be70a530b2e661da72ca4d42129d476135defc1139cd0a8b993 not found: ID does not exist" containerID="ab36eb187e9b9be70a530b2e661da72ca4d42129d476135defc1139cd0a8b993"
Jul 24 16:14:45.783036 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.782991    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab36eb187e9b9be70a530b2e661da72ca4d42129d476135defc1139cd0a8b993"} err="failed to get container status \"ab36eb187e9b9be70a530b2e661da72ca4d42129d476135defc1139cd0a8b993\": rpc error: code = NotFound desc = could not find container \"ab36eb187e9b9be70a530b2e661da72ca4d42129d476135defc1139cd0a8b993\": container with ID starting with ab36eb187e9b9be70a530b2e661da72ca4d42129d476135defc1139cd0a8b993 not found: ID does not exist"
Jul 24 16:14:45.783036 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.783008    2572 scope.go:117] "RemoveContainer" containerID="e1cdaa9fd85dca9a9d17580534185c47218793153b50eec300109c39c29d8146"
Jul 24 16:14:45.783220 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:14:45.783208    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1cdaa9fd85dca9a9d17580534185c47218793153b50eec300109c39c29d8146\": container with ID starting with e1cdaa9fd85dca9a9d17580534185c47218793153b50eec300109c39c29d8146 not found: ID does not exist" containerID="e1cdaa9fd85dca9a9d17580534185c47218793153b50eec300109c39c29d8146"
Jul 24 16:14:45.783269 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.783223    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1cdaa9fd85dca9a9d17580534185c47218793153b50eec300109c39c29d8146"} err="failed to get container status \"e1cdaa9fd85dca9a9d17580534185c47218793153b50eec300109c39c29d8146\": rpc error: code = NotFound desc = could not find container \"e1cdaa9fd85dca9a9d17580534185c47218793153b50eec300109c39c29d8146\": container with ID starting with e1cdaa9fd85dca9a9d17580534185c47218793153b50eec300109c39c29d8146 not found: ID does not exist"
Jul 24 16:14:45.788319 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.788297    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"]
Jul 24 16:14:45.793700 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:45.793679    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-c7a36573-gw-openshift-default-7465bdf8c6-wv498"]
Jul 24 16:14:47.223771 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:47.223737    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da2062ff-4747-4248-a0b6-9ad43f94c11e" path="/var/lib/kubelet/pods/da2062ff-4747-4248-a0b6-9ad43f94c11e/volumes"
Jul 24 16:14:58.218131 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:58.216704    2572 patch_prober.go:28] interesting pod/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": read tcp 10.132.0.2:41358->10.132.0.34:15021: read: connection reset by peer" start-of-body=
Jul 24 16:14:58.218131 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:58.216782    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn" podUID="1e85f31e-ae19-4373-9282-fdddb3ac273e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": read tcp 10.132.0.2:41358->10.132.0.34:15021: read: connection reset by peer"
Jul 24 16:14:58.833263 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:58.833231    2572 generic.go:356] "Generic (PLEG): container finished" podID="1e85f31e-ae19-4373-9282-fdddb3ac273e" containerID="65490cd11c1040d3741b42cd8ae388370bc25654df239461fde6eb33652ed8cf" exitCode=0
Jul 24 16:14:58.833431 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:58.833282    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn" event={"ID":"1e85f31e-ae19-4373-9282-fdddb3ac273e","Type":"ContainerDied","Data":"65490cd11c1040d3741b42cd8ae388370bc25654df239461fde6eb33652ed8cf"}
Jul 24 16:14:58.833696 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:58.833680    2572 scope.go:117] "RemoveContainer" containerID="65490cd11c1040d3741b42cd8ae388370bc25654df239461fde6eb33652ed8cf"
Jul 24 16:14:58.834690 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:58.834662    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:14:58.834752 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:58.834737    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:14:58.834789 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:58.834768    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:14:59.839299 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:14:59.839266    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn" event={"ID":"1e85f31e-ae19-4373-9282-fdddb3ac273e","Type":"ContainerStarted","Data":"db035cf7d6af0a6faf8bf297b6d57135b5b39da88e3cae78322f683b6987946d"}
Jul 24 16:15:00.402036 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:00.401995    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:15:00.402325 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:00.402294    2572 patch_prober.go:28] interesting pod/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Jul 24 16:15:00.402421 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:00.402357    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn" podUID="1e85f31e-ae19-4373-9282-fdddb3ac273e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Jul 24 16:15:01.402219 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:01.402177    2572 patch_prober.go:28] interesting pod/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Jul 24 16:15:01.402841 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:01.402807    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn" podUID="1e85f31e-ae19-4373-9282-fdddb3ac273e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Jul 24 16:15:02.402218 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:02.402169    2572 patch_prober.go:28] interesting pod/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Jul 24 16:15:02.402720 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:02.402241    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn" podUID="1e85f31e-ae19-4373-9282-fdddb3ac273e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Jul 24 16:15:03.402623 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:03.402573    2572 patch_prober.go:28] interesting pod/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Jul 24 16:15:03.403121 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:03.402648    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn" podUID="1e85f31e-ae19-4373-9282-fdddb3ac273e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Jul 24 16:15:04.406194 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:04.406162    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:15:04.406584 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:04.406437    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:15:04.407151 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:04.407133    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:15:21.220974 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:21.220911    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 16:15:21.228679 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:21.228653    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 16:15:25.247781 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:25.247739    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"]
Jul 24 16:15:25.248236 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:25.248062    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn" podUID="1e85f31e-ae19-4373-9282-fdddb3ac273e" containerName="istio-proxy" containerID="cri-o://db035cf7d6af0a6faf8bf297b6d57135b5b39da88e3cae78322f683b6987946d" gracePeriod=30
Jul 24 16:15:25.995119 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:25.995082    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"]
Jul 24 16:15:25.995576 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:25.995559    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da2062ff-4747-4248-a0b6-9ad43f94c11e" containerName="istio-proxy"
Jul 24 16:15:25.995627 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:25.995580    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="da2062ff-4747-4248-a0b6-9ad43f94c11e" containerName="istio-proxy"
Jul 24 16:15:25.995627 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:25.995602    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e523c92e-73a8-4ef0-ab0a-2accf416750f" containerName="istio-proxy"
Jul 24 16:15:25.995627 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:25.995611    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e523c92e-73a8-4ef0-ab0a-2accf416750f" containerName="istio-proxy"
Jul 24 16:15:25.995722 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:25.995684    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="da2062ff-4747-4248-a0b6-9ad43f94c11e" containerName="istio-proxy"
Jul 24 16:15:25.995722 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:25.995698    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="da2062ff-4747-4248-a0b6-9ad43f94c11e" containerName="istio-proxy"
Jul 24 16:15:25.995722 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:25.995706    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e523c92e-73a8-4ef0-ab0a-2accf416750f" containerName="istio-proxy"
Jul 24 16:15:26.000159 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.000131    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.002963 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.002936    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-188903f5-openshift-default-dockercfg-75mlj\""
Jul 24 16:15:26.010601 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.010571    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"]
Jul 24 16:15:26.116939 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.116887    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1200346-8152-4b56-8abb-5aced1d6da9c-istiod-ca-cert\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.117115 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.116945    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-workload-socket\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.117115 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.117005    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-envoy\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.117115 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.117024    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-data\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.117222 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.117125    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-workload-certs\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.117222 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.117159    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-podinfo\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.117222 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.117201    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-token\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.117317 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.117248    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-credential-socket\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.117317 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.117288    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4p92\" (UniqueName: \"kubernetes.io/projected/f1200346-8152-4b56-8abb-5aced1d6da9c-kube-api-access-j4p92\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.218299 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.218259    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-workload-socket\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.218510 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.218312    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-envoy\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.218510 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.218339    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-data\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.218510 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.218389    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-workload-certs\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.218510 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.218414    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-podinfo\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.218734 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.218607    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-token\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.218734 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.218662    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-credential-socket\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.218734 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.218703    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4p92\" (UniqueName: \"kubernetes.io/projected/f1200346-8152-4b56-8abb-5aced1d6da9c-kube-api-access-j4p92\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.218892 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.218729    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-workload-socket\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.218892 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.218767    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1200346-8152-4b56-8abb-5aced1d6da9c-istiod-ca-cert\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.218892 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.218800    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-data\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.218892 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.218848    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-workload-certs\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.219109 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.219023    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-credential-socket\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.219475 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.219374    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1200346-8152-4b56-8abb-5aced1d6da9c-istiod-ca-cert\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.221060 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.221039    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-envoy\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.221206 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.221189    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-podinfo\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.228775 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.228745    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-token\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.228906 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.228780    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4p92\" (UniqueName: \"kubernetes.io/projected/f1200346-8152-4b56-8abb-5aced1d6da9c-kube-api-access-j4p92\") pod \"e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") " pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.311912 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.311816    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:26.459931 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.459891    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"]
Jul 24 16:15:26.461493 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:15:26.461464    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1200346_8152_4b56_8abb_5aced1d6da9c.slice/crio-ed499e5d5e57260ebe3fa0a576da74bd4dfecca9b82cc8c1d9188b1c29d7a03b WatchSource:0}: Error finding container ed499e5d5e57260ebe3fa0a576da74bd4dfecca9b82cc8c1d9188b1c29d7a03b: Status 404 returned error can't find the container with id ed499e5d5e57260ebe3fa0a576da74bd4dfecca9b82cc8c1d9188b1c29d7a03b
Jul 24 16:15:26.469836 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.464090    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:15:26.469836 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.464250    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:15:26.469836 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.464341    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:15:26.962062 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.962023    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b" event={"ID":"f1200346-8152-4b56-8abb-5aced1d6da9c","Type":"ContainerStarted","Data":"d9633334b8709514a237392d772a88342d0d5092ffb5aac4615cc667ec4bba0c"}
Jul 24 16:15:26.962241 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.962102    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b" event={"ID":"f1200346-8152-4b56-8abb-5aced1d6da9c","Type":"ContainerStarted","Data":"ed499e5d5e57260ebe3fa0a576da74bd4dfecca9b82cc8c1d9188b1c29d7a03b"}
Jul 24 16:15:26.985321 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:26.985264    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b" podStartSLOduration=1.985246772 podStartE2EDuration="1.985246772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:15:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:15:26.983469846 +0000 UTC m=+1806.267797454" watchObservedRunningTime="2026-07-24 16:15:26.985246772 +0000 UTC m=+1806.269574446"
Jul 24 16:15:27.312649 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:27.312550    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:27.317586 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:27.317561    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:27.966654 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:27.966618    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:27.967719 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:27.967693    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:30.499144 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.499114    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:15:30.556764 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.556669    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-workload-certs\") pod \"1e85f31e-ae19-4373-9282-fdddb3ac273e\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") "
Jul 24 16:15:30.556764 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.556717    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-workload-socket\") pod \"1e85f31e-ae19-4373-9282-fdddb3ac273e\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") "
Jul 24 16:15:30.557040 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.556779    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-podinfo\") pod \"1e85f31e-ae19-4373-9282-fdddb3ac273e\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") "
Jul 24 16:15:30.557040 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.556801    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-envoy\") pod \"1e85f31e-ae19-4373-9282-fdddb3ac273e\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") "
Jul 24 16:15:30.557040 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.556841    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e85f31e-ae19-4373-9282-fdddb3ac273e-istiod-ca-cert\") pod \"1e85f31e-ae19-4373-9282-fdddb3ac273e\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") "
Jul 24 16:15:30.557040 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.556974    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9pqp\" (UniqueName: \"kubernetes.io/projected/1e85f31e-ae19-4373-9282-fdddb3ac273e-kube-api-access-c9pqp\") pod \"1e85f31e-ae19-4373-9282-fdddb3ac273e\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") "
Jul 24 16:15:30.557040 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.557008    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1e85f31e-ae19-4373-9282-fdddb3ac273e" (UID: "1e85f31e-ae19-4373-9282-fdddb3ac273e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:15:30.557040 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.557019    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-data\") pod \"1e85f31e-ae19-4373-9282-fdddb3ac273e\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") "
Jul 24 16:15:30.557378 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.557058    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-credential-socket\") pod \"1e85f31e-ae19-4373-9282-fdddb3ac273e\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") "
Jul 24 16:15:30.557378 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.557101    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-token\") pod \"1e85f31e-ae19-4373-9282-fdddb3ac273e\" (UID: \"1e85f31e-ae19-4373-9282-fdddb3ac273e\") "
Jul 24 16:15:30.557378 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.557126    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1e85f31e-ae19-4373-9282-fdddb3ac273e" (UID: "1e85f31e-ae19-4373-9282-fdddb3ac273e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:15:30.557378 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.557318    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e85f31e-ae19-4373-9282-fdddb3ac273e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1e85f31e-ae19-4373-9282-fdddb3ac273e" (UID: "1e85f31e-ae19-4373-9282-fdddb3ac273e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:15:30.557378 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.557334    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1e85f31e-ae19-4373-9282-fdddb3ac273e" (UID: "1e85f31e-ae19-4373-9282-fdddb3ac273e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:15:30.557592 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.557471    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-workload-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:15:30.557592 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.557489    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-workload-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:15:30.557592 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.557503    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e85f31e-ae19-4373-9282-fdddb3ac273e-istiod-ca-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:15:30.557592 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.557517    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-credential-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:15:30.558097 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.558072    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-data" (OuterVolumeSpecName: "istio-data") pod "1e85f31e-ae19-4373-9282-fdddb3ac273e" (UID: "1e85f31e-ae19-4373-9282-fdddb3ac273e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:15:30.559304 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.559273    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1e85f31e-ae19-4373-9282-fdddb3ac273e" (UID: "1e85f31e-ae19-4373-9282-fdddb3ac273e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:15:30.559425 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.559379    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-token" (OuterVolumeSpecName: "istio-token") pod "1e85f31e-ae19-4373-9282-fdddb3ac273e" (UID: "1e85f31e-ae19-4373-9282-fdddb3ac273e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:15:30.559677 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.559653    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e85f31e-ae19-4373-9282-fdddb3ac273e-kube-api-access-c9pqp" (OuterVolumeSpecName: "kube-api-access-c9pqp") pod "1e85f31e-ae19-4373-9282-fdddb3ac273e" (UID: "1e85f31e-ae19-4373-9282-fdddb3ac273e"). InnerVolumeSpecName "kube-api-access-c9pqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:15:30.559677 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.559663    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1e85f31e-ae19-4373-9282-fdddb3ac273e" (UID: "1e85f31e-ae19-4373-9282-fdddb3ac273e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 16:15:30.658905 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.658850    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-podinfo\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:15:30.658905 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.658893    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-envoy\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:15:30.658905 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.658903    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9pqp\" (UniqueName: \"kubernetes.io/projected/1e85f31e-ae19-4373-9282-fdddb3ac273e-kube-api-access-c9pqp\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:15:30.658905 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.658911    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-data\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:15:30.658905 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.658942    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e85f31e-ae19-4373-9282-fdddb3ac273e-istio-token\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:15:30.981836 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.981789    2572 generic.go:356] "Generic (PLEG): container finished" podID="1e85f31e-ae19-4373-9282-fdddb3ac273e" containerID="db035cf7d6af0a6faf8bf297b6d57135b5b39da88e3cae78322f683b6987946d" exitCode=0
Jul 24 16:15:30.982053 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.981847    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"
Jul 24 16:15:30.982053 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.981851    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn" event={"ID":"1e85f31e-ae19-4373-9282-fdddb3ac273e","Type":"ContainerDied","Data":"db035cf7d6af0a6faf8bf297b6d57135b5b39da88e3cae78322f683b6987946d"}
Jul 24 16:15:30.982053 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.981908    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn" event={"ID":"1e85f31e-ae19-4373-9282-fdddb3ac273e","Type":"ContainerDied","Data":"4a5ca623410c95fd0ab81f0c47792632e4268955a6213543cfd8fc4f5102ddd0"}
Jul 24 16:15:30.982053 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.981970    2572 scope.go:117] "RemoveContainer" containerID="db035cf7d6af0a6faf8bf297b6d57135b5b39da88e3cae78322f683b6987946d"
Jul 24 16:15:30.991098 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.991080    2572 scope.go:117] "RemoveContainer" containerID="65490cd11c1040d3741b42cd8ae388370bc25654df239461fde6eb33652ed8cf"
Jul 24 16:15:30.999180 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.999160    2572 scope.go:117] "RemoveContainer" containerID="db035cf7d6af0a6faf8bf297b6d57135b5b39da88e3cae78322f683b6987946d"
Jul 24 16:15:30.999458 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:15:30.999439    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db035cf7d6af0a6faf8bf297b6d57135b5b39da88e3cae78322f683b6987946d\": container with ID starting with db035cf7d6af0a6faf8bf297b6d57135b5b39da88e3cae78322f683b6987946d not found: ID does not exist" containerID="db035cf7d6af0a6faf8bf297b6d57135b5b39da88e3cae78322f683b6987946d"
Jul 24 16:15:30.999508 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.999466    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db035cf7d6af0a6faf8bf297b6d57135b5b39da88e3cae78322f683b6987946d"} err="failed to get container status \"db035cf7d6af0a6faf8bf297b6d57135b5b39da88e3cae78322f683b6987946d\": rpc error: code = NotFound desc = could not find container \"db035cf7d6af0a6faf8bf297b6d57135b5b39da88e3cae78322f683b6987946d\": container with ID starting with db035cf7d6af0a6faf8bf297b6d57135b5b39da88e3cae78322f683b6987946d not found: ID does not exist"
Jul 24 16:15:30.999508 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.999499    2572 scope.go:117] "RemoveContainer" containerID="65490cd11c1040d3741b42cd8ae388370bc25654df239461fde6eb33652ed8cf"
Jul 24 16:15:30.999715 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:15:30.999700    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65490cd11c1040d3741b42cd8ae388370bc25654df239461fde6eb33652ed8cf\": container with ID starting with 65490cd11c1040d3741b42cd8ae388370bc25654df239461fde6eb33652ed8cf not found: ID does not exist" containerID="65490cd11c1040d3741b42cd8ae388370bc25654df239461fde6eb33652ed8cf"
Jul 24 16:15:30.999761 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:30.999720    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65490cd11c1040d3741b42cd8ae388370bc25654df239461fde6eb33652ed8cf"} err="failed to get container status \"65490cd11c1040d3741b42cd8ae388370bc25654df239461fde6eb33652ed8cf\": rpc error: code = NotFound desc = could not find container \"65490cd11c1040d3741b42cd8ae388370bc25654df239461fde6eb33652ed8cf\": container with ID starting with 65490cd11c1040d3741b42cd8ae388370bc25654df239461fde6eb33652ed8cf not found: ID does not exist"
Jul 24 16:15:31.004955 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:31.004913    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"]
Jul 24 16:15:31.013501 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:31.013474    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-518181e2-openshift-default-59c74b8c78-sz6zn"]
Jul 24 16:15:31.225651 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:31.225604    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e85f31e-ae19-4373-9282-fdddb3ac273e" path="/var/lib/kubelet/pods/1e85f31e-ae19-4373-9282-fdddb3ac273e/volumes"
Jul 24 16:15:43.294079 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:43.293082    2572 patch_prober.go:28] interesting pod/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": read tcp 10.132.0.2:37216->10.132.0.35:15021: read: connection reset by peer" start-of-body=
Jul 24 16:15:43.294079 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:43.293151    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b" podUID="f1200346-8152-4b56-8abb-5aced1d6da9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": read tcp 10.132.0.2:37216->10.132.0.35:15021: read: connection reset by peer"
Jul 24 16:15:44.047248 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:44.047210    2572 generic.go:356] "Generic (PLEG): container finished" podID="f1200346-8152-4b56-8abb-5aced1d6da9c" containerID="d9633334b8709514a237392d772a88342d0d5092ffb5aac4615cc667ec4bba0c" exitCode=0
Jul 24 16:15:44.047423 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:44.047284    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b" event={"ID":"f1200346-8152-4b56-8abb-5aced1d6da9c","Type":"ContainerDied","Data":"d9633334b8709514a237392d772a88342d0d5092ffb5aac4615cc667ec4bba0c"}
Jul 24 16:15:44.047682 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:44.047668    2572 scope.go:117] "RemoveContainer" containerID="d9633334b8709514a237392d772a88342d0d5092ffb5aac4615cc667ec4bba0c"
Jul 24 16:15:44.048720 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:44.048696    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:15:44.048785 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:44.048778    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:15:44.048822 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:44.048811    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:15:45.059559 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:45.059513    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b" event={"ID":"f1200346-8152-4b56-8abb-5aced1d6da9c","Type":"ContainerStarted","Data":"49edaf6c88e585a2d4e43dc81550302a07962ad6ca7a93b97a3db28743e3c248"}
Jul 24 16:15:45.312173 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:45.312068    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:45.312744 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:45.312718    2572 patch_prober.go:28] interesting pod/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 24 16:15:45.312977 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:45.312951    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b" podUID="f1200346-8152-4b56-8abb-5aced1d6da9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 24 16:15:46.312815 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:46.312765    2572 patch_prober.go:28] interesting pod/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 24 16:15:46.313403 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:46.312839    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b" podUID="f1200346-8152-4b56-8abb-5aced1d6da9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 24 16:15:47.312406 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:47.312361    2572 patch_prober.go:28] interesting pod/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 24 16:15:47.312729 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:47.312697    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b" podUID="f1200346-8152-4b56-8abb-5aced1d6da9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 24 16:15:48.313450 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:48.313400    2572 patch_prober.go:28] interesting pod/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Jul 24 16:15:48.313989 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:48.313475    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b" podUID="f1200346-8152-4b56-8abb-5aced1d6da9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Jul 24 16:15:49.315973 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:49.315941    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:49.316402 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:49.316322    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:49.317023 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:49.317004    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:53.315758 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:53.315722    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"]
Jul 24 16:15:53.316337 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:53.315977    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b" podUID="f1200346-8152-4b56-8abb-5aced1d6da9c" containerName="istio-proxy" containerID="cri-o://49edaf6c88e585a2d4e43dc81550302a07962ad6ca7a93b97a3db28743e3c248" gracePeriod=30
Jul 24 16:15:58.577497 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.577433    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:58.621271 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.621225    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-token\") pod \"f1200346-8152-4b56-8abb-5aced1d6da9c\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") "
Jul 24 16:15:58.621447 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.621283    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-credential-socket\") pod \"f1200346-8152-4b56-8abb-5aced1d6da9c\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") "
Jul 24 16:15:58.621447 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.621313    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1200346-8152-4b56-8abb-5aced1d6da9c-istiod-ca-cert\") pod \"f1200346-8152-4b56-8abb-5aced1d6da9c\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") "
Jul 24 16:15:58.621544 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.621457    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-envoy\") pod \"f1200346-8152-4b56-8abb-5aced1d6da9c\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") "
Jul 24 16:15:58.621544 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.621487    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-workload-certs\") pod \"f1200346-8152-4b56-8abb-5aced1d6da9c\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") "
Jul 24 16:15:58.621544 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.621520    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4p92\" (UniqueName: \"kubernetes.io/projected/f1200346-8152-4b56-8abb-5aced1d6da9c-kube-api-access-j4p92\") pod \"f1200346-8152-4b56-8abb-5aced1d6da9c\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") "
Jul 24 16:15:58.621694 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.621559    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-workload-socket\") pod \"f1200346-8152-4b56-8abb-5aced1d6da9c\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") "
Jul 24 16:15:58.621694 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.621587    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-podinfo\") pod \"f1200346-8152-4b56-8abb-5aced1d6da9c\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") "
Jul 24 16:15:58.621694 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.621602    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f1200346-8152-4b56-8abb-5aced1d6da9c" (UID: "f1200346-8152-4b56-8abb-5aced1d6da9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:15:58.621694 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.621629    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-data\") pod \"f1200346-8152-4b56-8abb-5aced1d6da9c\" (UID: \"f1200346-8152-4b56-8abb-5aced1d6da9c\") "
Jul 24 16:15:58.621896 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.621694    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1200346-8152-4b56-8abb-5aced1d6da9c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f1200346-8152-4b56-8abb-5aced1d6da9c" (UID: "f1200346-8152-4b56-8abb-5aced1d6da9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:15:58.621896 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.621835    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f1200346-8152-4b56-8abb-5aced1d6da9c" (UID: "f1200346-8152-4b56-8abb-5aced1d6da9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:15:58.621896 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.621891    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-credential-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:15:58.622062 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.621909    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1200346-8152-4b56-8abb-5aced1d6da9c-istiod-ca-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:15:58.622501 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.622471    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f1200346-8152-4b56-8abb-5aced1d6da9c" (UID: "f1200346-8152-4b56-8abb-5aced1d6da9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:15:58.622783 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.622753    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-data" (OuterVolumeSpecName: "istio-data") pod "f1200346-8152-4b56-8abb-5aced1d6da9c" (UID: "f1200346-8152-4b56-8abb-5aced1d6da9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:15:58.623878 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.623850    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f1200346-8152-4b56-8abb-5aced1d6da9c" (UID: "f1200346-8152-4b56-8abb-5aced1d6da9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:15:58.624015 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.623894    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-token" (OuterVolumeSpecName: "istio-token") pod "f1200346-8152-4b56-8abb-5aced1d6da9c" (UID: "f1200346-8152-4b56-8abb-5aced1d6da9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:15:58.624374 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.624353    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1200346-8152-4b56-8abb-5aced1d6da9c-kube-api-access-j4p92" (OuterVolumeSpecName: "kube-api-access-j4p92") pod "f1200346-8152-4b56-8abb-5aced1d6da9c" (UID: "f1200346-8152-4b56-8abb-5aced1d6da9c"). InnerVolumeSpecName "kube-api-access-j4p92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:15:58.624374 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.624362    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f1200346-8152-4b56-8abb-5aced1d6da9c" (UID: "f1200346-8152-4b56-8abb-5aced1d6da9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 16:15:58.722967 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.722885    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-token\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:15:58.722967 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.722961    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-envoy\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:15:58.722967 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.722971    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-workload-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:15:58.723257 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.722979    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4p92\" (UniqueName: \"kubernetes.io/projected/f1200346-8152-4b56-8abb-5aced1d6da9c-kube-api-access-j4p92\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:15:58.723257 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.723002    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-workload-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:15:58.723257 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.723011    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-podinfo\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:15:58.723257 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:58.723019    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1200346-8152-4b56-8abb-5aced1d6da9c-istio-data\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:15:59.119963 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:59.119912    2572 generic.go:356] "Generic (PLEG): container finished" podID="f1200346-8152-4b56-8abb-5aced1d6da9c" containerID="49edaf6c88e585a2d4e43dc81550302a07962ad6ca7a93b97a3db28743e3c248" exitCode=0
Jul 24 16:15:59.120160 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:59.119993    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"
Jul 24 16:15:59.120160 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:59.120014    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b" event={"ID":"f1200346-8152-4b56-8abb-5aced1d6da9c","Type":"ContainerDied","Data":"49edaf6c88e585a2d4e43dc81550302a07962ad6ca7a93b97a3db28743e3c248"}
Jul 24 16:15:59.120160 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:59.120105    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b" event={"ID":"f1200346-8152-4b56-8abb-5aced1d6da9c","Type":"ContainerDied","Data":"ed499e5d5e57260ebe3fa0a576da74bd4dfecca9b82cc8c1d9188b1c29d7a03b"}
Jul 24 16:15:59.120160 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:59.120135    2572 scope.go:117] "RemoveContainer" containerID="49edaf6c88e585a2d4e43dc81550302a07962ad6ca7a93b97a3db28743e3c248"
Jul 24 16:15:59.129485 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:59.129464    2572 scope.go:117] "RemoveContainer" containerID="d9633334b8709514a237392d772a88342d0d5092ffb5aac4615cc667ec4bba0c"
Jul 24 16:15:59.137351 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:59.137331    2572 scope.go:117] "RemoveContainer" containerID="49edaf6c88e585a2d4e43dc81550302a07962ad6ca7a93b97a3db28743e3c248"
Jul 24 16:15:59.137628 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:15:59.137607    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49edaf6c88e585a2d4e43dc81550302a07962ad6ca7a93b97a3db28743e3c248\": container with ID starting with 49edaf6c88e585a2d4e43dc81550302a07962ad6ca7a93b97a3db28743e3c248 not found: ID does not exist" containerID="49edaf6c88e585a2d4e43dc81550302a07962ad6ca7a93b97a3db28743e3c248"
Jul 24 16:15:59.137694 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:59.137637    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49edaf6c88e585a2d4e43dc81550302a07962ad6ca7a93b97a3db28743e3c248"} err="failed to get container status \"49edaf6c88e585a2d4e43dc81550302a07962ad6ca7a93b97a3db28743e3c248\": rpc error: code = NotFound desc = could not find container \"49edaf6c88e585a2d4e43dc81550302a07962ad6ca7a93b97a3db28743e3c248\": container with ID starting with 49edaf6c88e585a2d4e43dc81550302a07962ad6ca7a93b97a3db28743e3c248 not found: ID does not exist"
Jul 24 16:15:59.137694 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:59.137655    2572 scope.go:117] "RemoveContainer" containerID="d9633334b8709514a237392d772a88342d0d5092ffb5aac4615cc667ec4bba0c"
Jul 24 16:15:59.137909 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:15:59.137892    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9633334b8709514a237392d772a88342d0d5092ffb5aac4615cc667ec4bba0c\": container with ID starting with d9633334b8709514a237392d772a88342d0d5092ffb5aac4615cc667ec4bba0c not found: ID does not exist" containerID="d9633334b8709514a237392d772a88342d0d5092ffb5aac4615cc667ec4bba0c"
Jul 24 16:15:59.137998 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:59.137931    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9633334b8709514a237392d772a88342d0d5092ffb5aac4615cc667ec4bba0c"} err="failed to get container status \"d9633334b8709514a237392d772a88342d0d5092ffb5aac4615cc667ec4bba0c\": rpc error: code = NotFound desc = could not find container \"d9633334b8709514a237392d772a88342d0d5092ffb5aac4615cc667ec4bba0c\": container with ID starting with d9633334b8709514a237392d772a88342d0d5092ffb5aac4615cc667ec4bba0c not found: ID does not exist"
Jul 24 16:15:59.149764 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:59.149728    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"]
Jul 24 16:15:59.157868 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:59.157838    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-188903f5-openshift-default-7dc86cd776-ldv4b"]
Jul 24 16:15:59.223586 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:15:59.223555    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1200346-8152-4b56-8abb-5aced1d6da9c" path="/var/lib/kubelet/pods/f1200346-8152-4b56-8abb-5aced1d6da9c/volumes"
Jul 24 16:17:32.819839 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.819794    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"]
Jul 24 16:17:32.820354 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.820126    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e85f31e-ae19-4373-9282-fdddb3ac273e" containerName="istio-proxy"
Jul 24 16:17:32.820354 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.820138    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e85f31e-ae19-4373-9282-fdddb3ac273e" containerName="istio-proxy"
Jul 24 16:17:32.820354 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.820150    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da2062ff-4747-4248-a0b6-9ad43f94c11e" containerName="istio-proxy"
Jul 24 16:17:32.820354 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.820156    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="da2062ff-4747-4248-a0b6-9ad43f94c11e" containerName="istio-proxy"
Jul 24 16:17:32.820354 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.820162    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e85f31e-ae19-4373-9282-fdddb3ac273e" containerName="istio-proxy"
Jul 24 16:17:32.820354 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.820168    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e85f31e-ae19-4373-9282-fdddb3ac273e" containerName="istio-proxy"
Jul 24 16:17:32.820354 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.820173    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1200346-8152-4b56-8abb-5aced1d6da9c" containerName="istio-proxy"
Jul 24 16:17:32.820354 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.820178    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1200346-8152-4b56-8abb-5aced1d6da9c" containerName="istio-proxy"
Jul 24 16:17:32.820354 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.820185    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1200346-8152-4b56-8abb-5aced1d6da9c" containerName="istio-proxy"
Jul 24 16:17:32.820354 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.820190    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1200346-8152-4b56-8abb-5aced1d6da9c" containerName="istio-proxy"
Jul 24 16:17:32.820354 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.820240    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e85f31e-ae19-4373-9282-fdddb3ac273e" containerName="istio-proxy"
Jul 24 16:17:32.820354 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.820248    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1200346-8152-4b56-8abb-5aced1d6da9c" containerName="istio-proxy"
Jul 24 16:17:32.820354 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.820254    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1200346-8152-4b56-8abb-5aced1d6da9c" containerName="istio-proxy"
Jul 24 16:17:32.823239 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.823217    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:32.826525 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.826498    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-383dbbf6-openshift-default-dockercfg-7tmqt\""
Jul 24 16:17:32.861027 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.860986    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"]
Jul 24 16:17:32.961109 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.961061    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-token\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:32.961109 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.961109    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pnj8\" (UniqueName: \"kubernetes.io/projected/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-kube-api-access-4pnj8\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:32.961357 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.961197    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istiod-ca-cert\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:32.961357 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.961241    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-data\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:32.961357 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.961272    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-credential-socket\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:32.961357 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.961297    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-workload-certs\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:32.961357 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.961340    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-envoy\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:32.961535 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.961363    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-podinfo\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:32.961535 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:32.961431    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-workload-socket\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.062470 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.062426    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-envoy\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.062675 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.062506    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-podinfo\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.062675 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.062580    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-workload-socket\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.062675 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.062615    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-token\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.062675 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.062639    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pnj8\" (UniqueName: \"kubernetes.io/projected/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-kube-api-access-4pnj8\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.062887 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.062701    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istiod-ca-cert\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.062887 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.062729    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-data\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.062887 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.062774    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-credential-socket\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.062887 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.062796    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-workload-certs\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.063142 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.063004    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-workload-socket\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.063224 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.063202    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-workload-certs\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.063419 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.063390    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-credential-socket\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.063547 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.063422    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-data\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.063680 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.063657    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istiod-ca-cert\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.065074 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.065052    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-envoy\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.065269 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.065249    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-podinfo\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.074447 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.074371    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-token\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.074868 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.074839    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pnj8\" (UniqueName: \"kubernetes.io/projected/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-kube-api-access-4pnj8\") pod \"e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") " pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.133239 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.133196    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:33.288351 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.288316    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"]
Jul 24 16:17:33.290942 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:17:33.290887    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98c9258e_7ccc_4cc1_b099_ae2b84c0b646.slice/crio-7bb0ec0a15f836df97a7d8bc0df0549d28dae91893dc8c82879861649c0dbb02 WatchSource:0}: Error finding container 7bb0ec0a15f836df97a7d8bc0df0549d28dae91893dc8c82879861649c0dbb02: Status 404 returned error can't find the container with id 7bb0ec0a15f836df97a7d8bc0df0549d28dae91893dc8c82879861649c0dbb02
Jul 24 16:17:33.293050 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.293019    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:17:33.293115 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.293102    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:17:33.293157 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.293132    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:17:33.517956 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.517909    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm" event={"ID":"98c9258e-7ccc-4cc1-b099-ae2b84c0b646","Type":"ContainerStarted","Data":"9dc859f13ac37834b640d66cb8207c0856e043872626e2f99d916943629d28cf"}
Jul 24 16:17:33.518124 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.517979    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm" event={"ID":"98c9258e-7ccc-4cc1-b099-ae2b84c0b646","Type":"ContainerStarted","Data":"7bb0ec0a15f836df97a7d8bc0df0549d28dae91893dc8c82879861649c0dbb02"}
Jul 24 16:17:33.547761 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:33.547696    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm" podStartSLOduration=1.547677621 podStartE2EDuration="1.547677621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:17:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:17:33.546626384 +0000 UTC m=+1932.830953992" watchObservedRunningTime="2026-07-24 16:17:33.547677621 +0000 UTC m=+1932.832005228"
Jul 24 16:17:34.134147 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:34.134095    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:34.138856 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:34.138828    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:34.523015 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:34.522902    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:34.524030 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:34.524012    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:49.939103 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:49.939057    2572 patch_prober.go:28] interesting pod/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": read tcp 10.132.0.2:43176->10.132.0.36:15021: read: connection reset by peer" start-of-body=
Jul 24 16:17:49.939528 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:49.939132    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm" podUID="98c9258e-7ccc-4cc1-b099-ae2b84c0b646" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": read tcp 10.132.0.2:43176->10.132.0.36:15021: read: connection reset by peer"
Jul 24 16:17:50.595971 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:50.595899    2572 generic.go:356] "Generic (PLEG): container finished" podID="98c9258e-7ccc-4cc1-b099-ae2b84c0b646" containerID="9dc859f13ac37834b640d66cb8207c0856e043872626e2f99d916943629d28cf" exitCode=0
Jul 24 16:17:50.596315 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:50.595984    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm" event={"ID":"98c9258e-7ccc-4cc1-b099-ae2b84c0b646","Type":"ContainerDied","Data":"9dc859f13ac37834b640d66cb8207c0856e043872626e2f99d916943629d28cf"}
Jul 24 16:17:50.596465 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:50.596448    2572 scope.go:117] "RemoveContainer" containerID="9dc859f13ac37834b640d66cb8207c0856e043872626e2f99d916943629d28cf"
Jul 24 16:17:50.597620 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:50.597591    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:17:50.597710 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:50.597693    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:17:50.597754 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:50.597742    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:17:51.605333 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:51.605286    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm" event={"ID":"98c9258e-7ccc-4cc1-b099-ae2b84c0b646","Type":"ContainerStarted","Data":"9a34672273e3c06908cddc6d40b598ab531854a65d1f74e5d3800f77541ae8fb"}
Jul 24 16:17:52.133524 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:52.133492    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:52.133764 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:52.133743    2572 patch_prober.go:28] interesting pod/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Jul 24 16:17:52.133814 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:52.133794    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm" podUID="98c9258e-7ccc-4cc1-b099-ae2b84c0b646" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Jul 24 16:17:53.134301 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:53.134264    2572 patch_prober.go:28] interesting pod/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Jul 24 16:17:53.134691 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:53.134327    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm" podUID="98c9258e-7ccc-4cc1-b099-ae2b84c0b646" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Jul 24 16:17:54.134566 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:54.134532    2572 patch_prober.go:28] interesting pod/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Jul 24 16:17:54.134962 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:54.134589    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm" podUID="98c9258e-7ccc-4cc1-b099-ae2b84c0b646" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Jul 24 16:17:55.134119 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:55.134077    2572 patch_prober.go:28] interesting pod/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Jul 24 16:17:55.134320 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:55.134149    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm" podUID="98c9258e-7ccc-4cc1-b099-ae2b84c0b646" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Jul 24 16:17:56.138623 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:56.138590    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:56.139132 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:56.138945    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:17:56.139564 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:17:56.139546    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:18:09.589010 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:09.588973    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"]
Jul 24 16:18:09.589558 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:09.589524    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm" podUID="98c9258e-7ccc-4cc1-b099-ae2b84c0b646" containerName="istio-proxy" containerID="cri-o://9a34672273e3c06908cddc6d40b598ab531854a65d1f74e5d3800f77541ae8fb" gracePeriod=30
Jul 24 16:18:11.140894 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:11.140857    2572 patch_prober.go:28] interesting pod/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 24 16:18:11.141316 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:11.140955    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm" podUID="98c9258e-7ccc-4cc1-b099-ae2b84c0b646" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 16:18:14.716094 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.716060    2572 generic.go:356] "Generic (PLEG): container finished" podID="98c9258e-7ccc-4cc1-b099-ae2b84c0b646" containerID="9a34672273e3c06908cddc6d40b598ab531854a65d1f74e5d3800f77541ae8fb" exitCode=0
Jul 24 16:18:14.716476 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.716135    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm" event={"ID":"98c9258e-7ccc-4cc1-b099-ae2b84c0b646","Type":"ContainerDied","Data":"9a34672273e3c06908cddc6d40b598ab531854a65d1f74e5d3800f77541ae8fb"}
Jul 24 16:18:14.716476 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.716250    2572 scope.go:117] "RemoveContainer" containerID="9dc859f13ac37834b640d66cb8207c0856e043872626e2f99d916943629d28cf"
Jul 24 16:18:14.846984 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.846961    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:18:14.915756 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.915721    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istiod-ca-cert\") pod \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") "
Jul 24 16:18:14.915960 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.915766    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-token\") pod \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") "
Jul 24 16:18:14.915960 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.915784    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pnj8\" (UniqueName: \"kubernetes.io/projected/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-kube-api-access-4pnj8\") pod \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") "
Jul 24 16:18:14.915960 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.915828    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-credential-socket\") pod \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") "
Jul 24 16:18:14.915960 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.915851    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-envoy\") pod \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") "
Jul 24 16:18:14.915960 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.915872    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-data\") pod \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") "
Jul 24 16:18:14.915960 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.915889    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-podinfo\") pod \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") "
Jul 24 16:18:14.916644 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.915965    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-workload-certs\") pod \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") "
Jul 24 16:18:14.916644 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.916016    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-workload-socket\") pod \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\" (UID: \"98c9258e-7ccc-4cc1-b099-ae2b84c0b646\") "
Jul 24 16:18:14.916644 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.916277    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "98c9258e-7ccc-4cc1-b099-ae2b84c0b646" (UID: "98c9258e-7ccc-4cc1-b099-ae2b84c0b646"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:18:14.916644 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.916463    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "98c9258e-7ccc-4cc1-b099-ae2b84c0b646" (UID: "98c9258e-7ccc-4cc1-b099-ae2b84c0b646"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:18:14.916644 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.916485    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "98c9258e-7ccc-4cc1-b099-ae2b84c0b646" (UID: "98c9258e-7ccc-4cc1-b099-ae2b84c0b646"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:18:14.917110 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.916863    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "98c9258e-7ccc-4cc1-b099-ae2b84c0b646" (UID: "98c9258e-7ccc-4cc1-b099-ae2b84c0b646"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:18:14.917230 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.917199    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-data" (OuterVolumeSpecName: "istio-data") pod "98c9258e-7ccc-4cc1-b099-ae2b84c0b646" (UID: "98c9258e-7ccc-4cc1-b099-ae2b84c0b646"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:18:14.918321 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.918288    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "98c9258e-7ccc-4cc1-b099-ae2b84c0b646" (UID: "98c9258e-7ccc-4cc1-b099-ae2b84c0b646"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 16:18:14.918321 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.918295    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-token" (OuterVolumeSpecName: "istio-token") pod "98c9258e-7ccc-4cc1-b099-ae2b84c0b646" (UID: "98c9258e-7ccc-4cc1-b099-ae2b84c0b646"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:18:14.918497 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.918348    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-kube-api-access-4pnj8" (OuterVolumeSpecName: "kube-api-access-4pnj8") pod "98c9258e-7ccc-4cc1-b099-ae2b84c0b646" (UID: "98c9258e-7ccc-4cc1-b099-ae2b84c0b646"). InnerVolumeSpecName "kube-api-access-4pnj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:18:14.918497 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:14.918374    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "98c9258e-7ccc-4cc1-b099-ae2b84c0b646" (UID: "98c9258e-7ccc-4cc1-b099-ae2b84c0b646"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:18:15.017110 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:15.017059    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-workload-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:18:15.017110 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:15.017101    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istiod-ca-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:18:15.017110 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:15.017110    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-token\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:18:15.017110 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:15.017118    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pnj8\" (UniqueName: \"kubernetes.io/projected/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-kube-api-access-4pnj8\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:18:15.017390 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:15.017129    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-credential-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:18:15.017390 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:15.017137    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-envoy\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:18:15.017390 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:15.017150    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-data\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:18:15.017390 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:15.017161    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-istio-podinfo\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:18:15.017390 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:15.017170    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c9258e-7ccc-4cc1-b099-ae2b84c0b646-workload-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:18:15.722132 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:15.722106    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"
Jul 24 16:18:15.722614 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:15.722103    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm" event={"ID":"98c9258e-7ccc-4cc1-b099-ae2b84c0b646","Type":"ContainerDied","Data":"7bb0ec0a15f836df97a7d8bc0df0549d28dae91893dc8c82879861649c0dbb02"}
Jul 24 16:18:15.722614 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:15.722238    2572 scope.go:117] "RemoveContainer" containerID="9a34672273e3c06908cddc6d40b598ab531854a65d1f74e5d3800f77541ae8fb"
Jul 24 16:18:15.739456 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:15.739429    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"]
Jul 24 16:18:15.746462 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:15.746436    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-383dbbf6-openshift-default-6fc8c94cbd-642tm"]
Jul 24 16:18:17.222500 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:17.222463    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98c9258e-7ccc-4cc1-b099-ae2b84c0b646" path="/var/lib/kubelet/pods/98c9258e-7ccc-4cc1-b099-ae2b84c0b646/volumes"
Jul 24 16:18:29.499447 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.499410    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"]
Jul 24 16:18:29.499913 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.499718    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98c9258e-7ccc-4cc1-b099-ae2b84c0b646" containerName="istio-proxy"
Jul 24 16:18:29.499913 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.499729    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="98c9258e-7ccc-4cc1-b099-ae2b84c0b646" containerName="istio-proxy"
Jul 24 16:18:29.499913 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.499735    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98c9258e-7ccc-4cc1-b099-ae2b84c0b646" containerName="istio-proxy"
Jul 24 16:18:29.499913 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.499741    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="98c9258e-7ccc-4cc1-b099-ae2b84c0b646" containerName="istio-proxy"
Jul 24 16:18:29.499913 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.499800    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="98c9258e-7ccc-4cc1-b099-ae2b84c0b646" containerName="istio-proxy"
Jul 24 16:18:29.499913 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.499809    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e85f31e-ae19-4373-9282-fdddb3ac273e" containerName="istio-proxy"
Jul 24 16:18:29.501679 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.501659    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.504984 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.504963    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f4ba5c28-openshift-default-dockercfg-5lq6v\""
Jul 24 16:18:29.521658 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.521624    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"]
Jul 24 16:18:29.647359 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.647317    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-workload-socket\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.647566 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.647383    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-podinfo\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.647566 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.647431    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzrmx\" (UniqueName: \"kubernetes.io/projected/79b6270a-851b-4277-a0fe-a3fbe11badda-kube-api-access-dzrmx\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.647566 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.647528    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-workload-certs\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.647834 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.647615    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-credential-socket\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.647834 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.647656    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79b6270a-851b-4277-a0fe-a3fbe11badda-istiod-ca-cert\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.647834 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.647699    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-token\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.647834 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.647748    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-data\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.647834 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.647789    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-envoy\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.748385 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.748349    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-podinfo\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.748385 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.748388    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzrmx\" (UniqueName: \"kubernetes.io/projected/79b6270a-851b-4277-a0fe-a3fbe11badda-kube-api-access-dzrmx\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.748653 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.748410    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-workload-certs\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.748653 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.748440    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-credential-socket\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.748653 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.748460    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79b6270a-851b-4277-a0fe-a3fbe11badda-istiod-ca-cert\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.748653 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.748486    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-token\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.748653 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.748508    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-data\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.748653 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.748547    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-envoy\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.748653 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.748573    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-workload-socket\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.749035 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.748844    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-credential-socket\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.749035 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.748866    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-workload-certs\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.749250 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.749179    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-workload-socket\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.749440 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.749307    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79b6270a-851b-4277-a0fe-a3fbe11badda-istiod-ca-cert\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.749440 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.749430    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-data\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.751079 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.751022    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-envoy\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.751196 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.751179    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-podinfo\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.759034 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.759009    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzrmx\" (UniqueName: \"kubernetes.io/projected/79b6270a-851b-4277-a0fe-a3fbe11badda-kube-api-access-dzrmx\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.759830 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.759813    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-token\") pod \"e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") " pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.812322 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.812284    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:29.971309 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.971273    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"]
Jul 24 16:18:29.976128 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:18:29.976102    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79b6270a_851b_4277_a0fe_a3fbe11badda.slice/crio-234bce41d808fd26b0fe5f4076da1a7e98cabfe7a79e5a459ff5483500554b47 WatchSource:0}: Error finding container 234bce41d808fd26b0fe5f4076da1a7e98cabfe7a79e5a459ff5483500554b47: Status 404 returned error can't find the container with id 234bce41d808fd26b0fe5f4076da1a7e98cabfe7a79e5a459ff5483500554b47
Jul 24 16:18:29.978277 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.978246    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:18:29.978379 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.978326    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:18:29.978379 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:29.978372    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:18:30.791047 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:30.791012    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn" event={"ID":"79b6270a-851b-4277-a0fe-a3fbe11badda","Type":"ContainerStarted","Data":"d9b157a772c5230296cb1a470ff70f98f1de6aebe23813f2572bd2613c52f40f"}
Jul 24 16:18:30.791418 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:30.791065    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn" event={"ID":"79b6270a-851b-4277-a0fe-a3fbe11badda","Type":"ContainerStarted","Data":"234bce41d808fd26b0fe5f4076da1a7e98cabfe7a79e5a459ff5483500554b47"}
Jul 24 16:18:30.813050 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:30.813023    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:31.818239 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:31.818205    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:31.849956 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:31.849890    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn" podStartSLOduration=2.849872433 podStartE2EDuration="2.849872433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:18:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:18:30.836445367 +0000 UTC m=+1990.120772974" watchObservedRunningTime="2026-07-24 16:18:31.849872433 +0000 UTC m=+1991.134200040"
Jul 24 16:18:32.799829 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:32.799796    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:32.800687 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:32.800667    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:47.874797 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:47.874305    2572 patch_prober.go:28] interesting pod/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": read tcp 10.132.0.2:59092->10.132.0.37:15021: read: connection reset by peer" start-of-body=
Jul 24 16:18:47.874797 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:47.874375    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn" podUID="79b6270a-851b-4277-a0fe-a3fbe11badda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": read tcp 10.132.0.2:59092->10.132.0.37:15021: read: connection reset by peer"
Jul 24 16:18:48.871734 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:48.871696    2572 generic.go:356] "Generic (PLEG): container finished" podID="79b6270a-851b-4277-a0fe-a3fbe11badda" containerID="d9b157a772c5230296cb1a470ff70f98f1de6aebe23813f2572bd2613c52f40f" exitCode=0
Jul 24 16:18:48.871933 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:48.871769    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn" event={"ID":"79b6270a-851b-4277-a0fe-a3fbe11badda","Type":"ContainerDied","Data":"d9b157a772c5230296cb1a470ff70f98f1de6aebe23813f2572bd2613c52f40f"}
Jul 24 16:18:48.872238 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:48.872220    2572 scope.go:117] "RemoveContainer" containerID="d9b157a772c5230296cb1a470ff70f98f1de6aebe23813f2572bd2613c52f40f"
Jul 24 16:18:48.873388 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:48.873359    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:18:48.873451 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:48.873430    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:18:48.873492 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:48.873466    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:18:49.025400 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.025357    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"]
Jul 24 16:18:49.025911 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.025818    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="98c9258e-7ccc-4cc1-b099-ae2b84c0b646" containerName="istio-proxy"
Jul 24 16:18:49.027790 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.027764    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.031150 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.030900    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f4ba5c28\"/\"openshift-service-ca.crt\""
Jul 24 16:18:49.031328 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.031298    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f4ba5c28\"/\"default-dockercfg-krwn6\""
Jul 24 16:18:49.031532 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.031512    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f4ba5c28\"/\"kube-root-ca.crt\""
Jul 24 16:18:49.031954 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.031897    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f4ba5c28\"/\"e2e-lifecycle-model-f4ba5c28-kserve-self-signed-certs\""
Jul 24 16:18:49.053662 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.053629    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"]
Jul 24 16:18:49.118046 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.118004    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgtf7\" (UniqueName: \"kubernetes.io/projected/cceea788-487d-41ea-a543-a966750d6a25-kube-api-access-tgtf7\") pod \"e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") " pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.118251 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.118080    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cceea788-487d-41ea-a543-a966750d6a25-tls-certs\") pod \"e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") " pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.118251 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.118117    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-home\") pod \"e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") " pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.118251 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.118148    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-tmp-dir\") pod \"e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") " pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.118251 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.118214    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-dshm\") pod \"e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") " pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.118444 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.118290    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-model-cache\") pod \"e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") " pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.219113 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.219073    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-model-cache\") pod \"e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") " pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.219324 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.219124    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgtf7\" (UniqueName: \"kubernetes.io/projected/cceea788-487d-41ea-a543-a966750d6a25-kube-api-access-tgtf7\") pod \"e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") " pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.219324 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.219165    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cceea788-487d-41ea-a543-a966750d6a25-tls-certs\") pod \"e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") " pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.219324 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.219194    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-home\") pod \"e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") " pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.219324 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.219226    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-tmp-dir\") pod \"e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") " pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.219324 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.219281    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-dshm\") pod \"e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") " pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.219660 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.219493    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-model-cache\") pod \"e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") " pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.219660 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.219649    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-home\") pod \"e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") " pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.219769 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.219687    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-tmp-dir\") pod \"e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") " pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.221815 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.221788    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-dshm\") pod \"e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") " pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.222146 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.222125    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cceea788-487d-41ea-a543-a966750d6a25-tls-certs\") pod \"e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") " pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.231657 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.231618    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgtf7\" (UniqueName: \"kubernetes.io/projected/cceea788-487d-41ea-a543-a966750d6a25-kube-api-access-tgtf7\") pod \"e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") " pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.339316 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.339276    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:49.493980 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.492374    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"]
Jul 24 16:18:49.878620 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.878525    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn" event={"ID":"79b6270a-851b-4277-a0fe-a3fbe11badda","Type":"ContainerStarted","Data":"c7d35d8e2fb3e22bb8da0e5a7f9e2702692c78423f4b6839395dcc27a144039c"}
Jul 24 16:18:49.880107 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.880078    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw" event={"ID":"cceea788-487d-41ea-a543-a966750d6a25","Type":"ContainerStarted","Data":"d0eab8f434dd6bb9c9dab8220a6d7435af7344bcdf8f4d5ab471e824d3e79e86"}
Jul 24 16:18:49.880236 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:49.880119    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw" event={"ID":"cceea788-487d-41ea-a543-a966750d6a25","Type":"ContainerStarted","Data":"4ab63afe15183931f8e6cb1ccbe3f59e530ae69dc5dc3fddc2e839600bba1ca5"}
Jul 24 16:18:50.813031 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:50.812979    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:50.813660 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:50.813135    2572 patch_prober.go:28] interesting pod/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 24 16:18:50.813869 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:50.813839    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn" podUID="79b6270a-851b-4277-a0fe-a3fbe11badda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 24 16:18:51.813706 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:51.813663    2572 patch_prober.go:28] interesting pod/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 24 16:18:51.814404 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:51.814369    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn" podUID="79b6270a-851b-4277-a0fe-a3fbe11badda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 24 16:18:52.813090 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:52.813048    2572 patch_prober.go:28] interesting pod/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 24 16:18:52.813494 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:52.813417    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn" podUID="79b6270a-851b-4277-a0fe-a3fbe11badda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 24 16:18:53.813680 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:53.813638    2572 patch_prober.go:28] interesting pod/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body=
Jul 24 16:18:53.814331 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:53.814298    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn" podUID="79b6270a-851b-4277-a0fe-a3fbe11badda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused"
Jul 24 16:18:54.816617 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:54.816584    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:54.817005 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:54.816948    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:54.817623 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:54.817604    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:18:54.845643 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:54.845591    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw" podStartSLOduration=6.845575601 podStartE2EDuration="6.845575601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:18:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:18:49.938148401 +0000 UTC m=+2009.222476011" watchObservedRunningTime="2026-07-24 16:18:54.845575601 +0000 UTC m=+2014.129903253"
Jul 24 16:18:59.340220 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:59.340179    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:59.340671 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:59.340436    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:59.352856 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:59.352830    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:18:59.938699 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:18:59.938672    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:19:45.951033 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:19:45.950996    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"]
Jul 24 16:20:00.970236 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:00.970174    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw" podUID="cceea788-487d-41ea-a543-a966750d6a25" containerName="main" containerID="cri-o://d0eab8f434dd6bb9c9dab8220a6d7435af7344bcdf8f4d5ab471e824d3e79e86" gracePeriod=45
Jul 24 16:20:01.205227 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.205199    2572 generic.go:356] "Generic (PLEG): container finished" podID="cceea788-487d-41ea-a543-a966750d6a25" containerID="d0eab8f434dd6bb9c9dab8220a6d7435af7344bcdf8f4d5ab471e824d3e79e86" exitCode=0
Jul 24 16:20:01.205369 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.205273    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw" event={"ID":"cceea788-487d-41ea-a543-a966750d6a25","Type":"ContainerDied","Data":"d0eab8f434dd6bb9c9dab8220a6d7435af7344bcdf8f4d5ab471e824d3e79e86"}
Jul 24 16:20:01.220931 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.220846    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:20:01.358644 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.358610    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cceea788-487d-41ea-a543-a966750d6a25-tls-certs\") pod \"cceea788-487d-41ea-a543-a966750d6a25\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") "
Jul 24 16:20:01.358833 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.358659    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgtf7\" (UniqueName: \"kubernetes.io/projected/cceea788-487d-41ea-a543-a966750d6a25-kube-api-access-tgtf7\") pod \"cceea788-487d-41ea-a543-a966750d6a25\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") "
Jul 24 16:20:01.358833 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.358702    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-model-cache\") pod \"cceea788-487d-41ea-a543-a966750d6a25\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") "
Jul 24 16:20:01.358833 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.358746    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-dshm\") pod \"cceea788-487d-41ea-a543-a966750d6a25\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") "
Jul 24 16:20:01.358833 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.358774    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-tmp-dir\") pod \"cceea788-487d-41ea-a543-a966750d6a25\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") "
Jul 24 16:20:01.358833 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.358804    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-home\") pod \"cceea788-487d-41ea-a543-a966750d6a25\" (UID: \"cceea788-487d-41ea-a543-a966750d6a25\") "
Jul 24 16:20:01.359146 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.359116    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-home" (OuterVolumeSpecName: "home") pod "cceea788-487d-41ea-a543-a966750d6a25" (UID: "cceea788-487d-41ea-a543-a966750d6a25"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:20:01.359200 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.359140    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cceea788-487d-41ea-a543-a966750d6a25" (UID: "cceea788-487d-41ea-a543-a966750d6a25"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:20:01.359200 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.359123    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-model-cache" (OuterVolumeSpecName: "model-cache") pod "cceea788-487d-41ea-a543-a966750d6a25" (UID: "cceea788-487d-41ea-a543-a966750d6a25"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:20:01.361205 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.361175    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cceea788-487d-41ea-a543-a966750d6a25-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cceea788-487d-41ea-a543-a966750d6a25" (UID: "cceea788-487d-41ea-a543-a966750d6a25"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:20:01.361328 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.361246    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cceea788-487d-41ea-a543-a966750d6a25-kube-api-access-tgtf7" (OuterVolumeSpecName: "kube-api-access-tgtf7") pod "cceea788-487d-41ea-a543-a966750d6a25" (UID: "cceea788-487d-41ea-a543-a966750d6a25"). InnerVolumeSpecName "kube-api-access-tgtf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:20:01.361328 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.361252    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-dshm" (OuterVolumeSpecName: "dshm") pod "cceea788-487d-41ea-a543-a966750d6a25" (UID: "cceea788-487d-41ea-a543-a966750d6a25"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:20:01.460103 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.460064    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cceea788-487d-41ea-a543-a966750d6a25-tls-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:20:01.460103 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.460097    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgtf7\" (UniqueName: \"kubernetes.io/projected/cceea788-487d-41ea-a543-a966750d6a25-kube-api-access-tgtf7\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:20:01.460103 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.460111    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-model-cache\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:20:01.460355 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.460124    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-dshm\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:20:01.460355 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.460135    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-tmp-dir\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:20:01.460355 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:01.460145    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cceea788-487d-41ea-a543-a966750d6a25-home\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:20:02.211071 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:02.211030    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw" event={"ID":"cceea788-487d-41ea-a543-a966750d6a25","Type":"ContainerDied","Data":"4ab63afe15183931f8e6cb1ccbe3f59e530ae69dc5dc3fddc2e839600bba1ca5"}
Jul 24 16:20:02.211071 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:02.211057    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"
Jul 24 16:20:02.211595 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:02.211093    2572 scope.go:117] "RemoveContainer" containerID="d0eab8f434dd6bb9c9dab8220a6d7435af7344bcdf8f4d5ab471e824d3e79e86"
Jul 24 16:20:02.242725 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:02.242692    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"]
Jul 24 16:20:02.253140 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:02.253117    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-f4ba5c28/e2e-lifecycle-model-f4ba5c28-kserve-8477b85f5-cqhrw"]
Jul 24 16:20:03.222250 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:03.222218    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cceea788-487d-41ea-a543-a966750d6a25" path="/var/lib/kubelet/pods/cceea788-487d-41ea-a543-a966750d6a25/volumes"
Jul 24 16:20:21.246254 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:21.246225    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 16:20:21.253703 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:21.253678    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 16:20:30.771405 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:30.771361    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"]
Jul 24 16:20:30.772022 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:30.771645    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn" podUID="79b6270a-851b-4277-a0fe-a3fbe11badda" containerName="istio-proxy" containerID="cri-o://c7d35d8e2fb3e22bb8da0e5a7f9e2702692c78423f4b6839395dcc27a144039c" gracePeriod=30
Jul 24 16:20:36.023565 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.023538    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:20:36.058877 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.058778    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-data\") pod \"79b6270a-851b-4277-a0fe-a3fbe11badda\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") "
Jul 24 16:20:36.058877 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.058820    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzrmx\" (UniqueName: \"kubernetes.io/projected/79b6270a-851b-4277-a0fe-a3fbe11badda-kube-api-access-dzrmx\") pod \"79b6270a-851b-4277-a0fe-a3fbe11badda\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") "
Jul 24 16:20:36.058877 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.058862    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-podinfo\") pod \"79b6270a-851b-4277-a0fe-a3fbe11badda\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") "
Jul 24 16:20:36.059170 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.058886    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-workload-certs\") pod \"79b6270a-851b-4277-a0fe-a3fbe11badda\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") "
Jul 24 16:20:36.059170 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.058912    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-credential-socket\") pod \"79b6270a-851b-4277-a0fe-a3fbe11badda\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") "
Jul 24 16:20:36.059170 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.058956    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79b6270a-851b-4277-a0fe-a3fbe11badda-istiod-ca-cert\") pod \"79b6270a-851b-4277-a0fe-a3fbe11badda\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") "
Jul 24 16:20:36.059325 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.059195    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "79b6270a-851b-4277-a0fe-a3fbe11badda" (UID: "79b6270a-851b-4277-a0fe-a3fbe11badda"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:20:36.059325 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.059277    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "79b6270a-851b-4277-a0fe-a3fbe11badda" (UID: "79b6270a-851b-4277-a0fe-a3fbe11badda"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:20:36.059440 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.059341    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-token\") pod \"79b6270a-851b-4277-a0fe-a3fbe11badda\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") "
Jul 24 16:20:36.059440 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.059424    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79b6270a-851b-4277-a0fe-a3fbe11badda-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "79b6270a-851b-4277-a0fe-a3fbe11badda" (UID: "79b6270a-851b-4277-a0fe-a3fbe11badda"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:20:36.059545 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.059432    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-envoy\") pod \"79b6270a-851b-4277-a0fe-a3fbe11badda\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") "
Jul 24 16:20:36.059545 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.059486    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-workload-socket\") pod \"79b6270a-851b-4277-a0fe-a3fbe11badda\" (UID: \"79b6270a-851b-4277-a0fe-a3fbe11badda\") "
Jul 24 16:20:36.059851 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.059771    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-workload-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:20:36.059851 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.059794    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-credential-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:20:36.059851 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.059808    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79b6270a-851b-4277-a0fe-a3fbe11badda-istiod-ca-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:20:36.060159 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.059990    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "79b6270a-851b-4277-a0fe-a3fbe11badda" (UID: "79b6270a-851b-4277-a0fe-a3fbe11badda"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:20:36.060159 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.060001    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-data" (OuterVolumeSpecName: "istio-data") pod "79b6270a-851b-4277-a0fe-a3fbe11badda" (UID: "79b6270a-851b-4277-a0fe-a3fbe11badda"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:20:36.062066 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.062041    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79b6270a-851b-4277-a0fe-a3fbe11badda-kube-api-access-dzrmx" (OuterVolumeSpecName: "kube-api-access-dzrmx") pod "79b6270a-851b-4277-a0fe-a3fbe11badda" (UID: "79b6270a-851b-4277-a0fe-a3fbe11badda"). InnerVolumeSpecName "kube-api-access-dzrmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:20:36.062185 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.062086    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-token" (OuterVolumeSpecName: "istio-token") pod "79b6270a-851b-4277-a0fe-a3fbe11badda" (UID: "79b6270a-851b-4277-a0fe-a3fbe11badda"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:20:36.062185 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.062089    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "79b6270a-851b-4277-a0fe-a3fbe11badda" (UID: "79b6270a-851b-4277-a0fe-a3fbe11badda"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 16:20:36.062398 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.062370    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "79b6270a-851b-4277-a0fe-a3fbe11badda" (UID: "79b6270a-851b-4277-a0fe-a3fbe11badda"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:20:36.161003 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.160968    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-podinfo\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:20:36.161003 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.160995    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-token\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:20:36.161003 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.161005    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-envoy\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:20:36.161003 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.161012    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-workload-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:20:36.161285 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.161020    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79b6270a-851b-4277-a0fe-a3fbe11badda-istio-data\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:20:36.161285 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.161029    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzrmx\" (UniqueName: \"kubernetes.io/projected/79b6270a-851b-4277-a0fe-a3fbe11badda-kube-api-access-dzrmx\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:20:36.366381 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.366282    2572 generic.go:356] "Generic (PLEG): container finished" podID="79b6270a-851b-4277-a0fe-a3fbe11badda" containerID="c7d35d8e2fb3e22bb8da0e5a7f9e2702692c78423f4b6839395dcc27a144039c" exitCode=0
Jul 24 16:20:36.366381 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.366325    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"
Jul 24 16:20:36.366381 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.366375    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn" event={"ID":"79b6270a-851b-4277-a0fe-a3fbe11badda","Type":"ContainerDied","Data":"c7d35d8e2fb3e22bb8da0e5a7f9e2702692c78423f4b6839395dcc27a144039c"}
Jul 24 16:20:36.366598 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.366425    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn" event={"ID":"79b6270a-851b-4277-a0fe-a3fbe11badda","Type":"ContainerDied","Data":"234bce41d808fd26b0fe5f4076da1a7e98cabfe7a79e5a459ff5483500554b47"}
Jul 24 16:20:36.366598 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.366446    2572 scope.go:117] "RemoveContainer" containerID="c7d35d8e2fb3e22bb8da0e5a7f9e2702692c78423f4b6839395dcc27a144039c"
Jul 24 16:20:36.375500 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.375479    2572 scope.go:117] "RemoveContainer" containerID="d9b157a772c5230296cb1a470ff70f98f1de6aebe23813f2572bd2613c52f40f"
Jul 24 16:20:36.383878 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.383858    2572 scope.go:117] "RemoveContainer" containerID="c7d35d8e2fb3e22bb8da0e5a7f9e2702692c78423f4b6839395dcc27a144039c"
Jul 24 16:20:36.384187 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:20:36.384164    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7d35d8e2fb3e22bb8da0e5a7f9e2702692c78423f4b6839395dcc27a144039c\": container with ID starting with c7d35d8e2fb3e22bb8da0e5a7f9e2702692c78423f4b6839395dcc27a144039c not found: ID does not exist" containerID="c7d35d8e2fb3e22bb8da0e5a7f9e2702692c78423f4b6839395dcc27a144039c"
Jul 24 16:20:36.384238 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.384198    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7d35d8e2fb3e22bb8da0e5a7f9e2702692c78423f4b6839395dcc27a144039c"} err="failed to get container status \"c7d35d8e2fb3e22bb8da0e5a7f9e2702692c78423f4b6839395dcc27a144039c\": rpc error: code = NotFound desc = could not find container \"c7d35d8e2fb3e22bb8da0e5a7f9e2702692c78423f4b6839395dcc27a144039c\": container with ID starting with c7d35d8e2fb3e22bb8da0e5a7f9e2702692c78423f4b6839395dcc27a144039c not found: ID does not exist"
Jul 24 16:20:36.384238 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.384216    2572 scope.go:117] "RemoveContainer" containerID="d9b157a772c5230296cb1a470ff70f98f1de6aebe23813f2572bd2613c52f40f"
Jul 24 16:20:36.384459 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:20:36.384439    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9b157a772c5230296cb1a470ff70f98f1de6aebe23813f2572bd2613c52f40f\": container with ID starting with d9b157a772c5230296cb1a470ff70f98f1de6aebe23813f2572bd2613c52f40f not found: ID does not exist" containerID="d9b157a772c5230296cb1a470ff70f98f1de6aebe23813f2572bd2613c52f40f"
Jul 24 16:20:36.384506 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.384466    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9b157a772c5230296cb1a470ff70f98f1de6aebe23813f2572bd2613c52f40f"} err="failed to get container status \"d9b157a772c5230296cb1a470ff70f98f1de6aebe23813f2572bd2613c52f40f\": rpc error: code = NotFound desc = could not find container \"d9b157a772c5230296cb1a470ff70f98f1de6aebe23813f2572bd2613c52f40f\": container with ID starting with d9b157a772c5230296cb1a470ff70f98f1de6aebe23813f2572bd2613c52f40f not found: ID does not exist"
Jul 24 16:20:36.390253 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.390222    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"]
Jul 24 16:20:36.398503 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:36.398476    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f4ba5c28-openshift-default-869776b9dd-xqxhn"]
Jul 24 16:20:37.223285 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:20:37.223247    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79b6270a-851b-4277-a0fe-a3fbe11badda" path="/var/lib/kubelet/pods/79b6270a-851b-4277-a0fe-a3fbe11badda/volumes"
Jul 24 16:21:27.497513 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.497470    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"]
Jul 24 16:21:27.497910 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.497793    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79b6270a-851b-4277-a0fe-a3fbe11badda" containerName="istio-proxy"
Jul 24 16:21:27.497910 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.497805    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="79b6270a-851b-4277-a0fe-a3fbe11badda" containerName="istio-proxy"
Jul 24 16:21:27.497910 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.497812    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cceea788-487d-41ea-a543-a966750d6a25" containerName="main"
Jul 24 16:21:27.497910 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.497820    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cceea788-487d-41ea-a543-a966750d6a25" containerName="main"
Jul 24 16:21:27.497910 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.497835    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79b6270a-851b-4277-a0fe-a3fbe11badda" containerName="istio-proxy"
Jul 24 16:21:27.497910 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.497843    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="79b6270a-851b-4277-a0fe-a3fbe11badda" containerName="istio-proxy"
Jul 24 16:21:27.497910 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.497899    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="79b6270a-851b-4277-a0fe-a3fbe11badda" containerName="istio-proxy"
Jul 24 16:21:27.497910 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.497909    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cceea788-487d-41ea-a543-a966750d6a25" containerName="main"
Jul 24 16:21:27.498186 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.497938    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="79b6270a-851b-4277-a0fe-a3fbe11badda" containerName="istio-proxy"
Jul 24 16:21:27.500676 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.500654    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.503557 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.503537    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-57d2c2-openshift-default-dockercfg-dvl7g\""
Jul 24 16:21:27.508146 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.508125    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-credential-socket\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.508261 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.508159    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-podinfo\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.508261 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.508184    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-workload-socket\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.508261 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.508248    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfl48\" (UniqueName: \"kubernetes.io/projected/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-kube-api-access-qfl48\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.508406 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.508295    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-envoy\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.508406 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.508340    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-workload-certs\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.508406 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.508367    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-data\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.508406 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.508382    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istiod-ca-cert\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.508610 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.508441    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-token\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.515020 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.514992    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"]
Jul 24 16:21:27.609763 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.609720    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfl48\" (UniqueName: \"kubernetes.io/projected/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-kube-api-access-qfl48\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.610001 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.609830    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-envoy\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.610001 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.609909    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-workload-certs\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.610001 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.609958    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-data\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.610001 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.609984    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istiod-ca-cert\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.610250 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.610010    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-token\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.610250 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.610059    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-credential-socket\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.610250 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.610096    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-podinfo\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.610250 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.610123    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-workload-socket\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.610510 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.610457    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-data\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.610644 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.610540    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-workload-socket\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.610720 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.610665    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-credential-socket\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.610813 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.610792    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istiod-ca-cert\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.610872 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.610863    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-workload-certs\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.612397 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.612377    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-envoy\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.612938 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.612868    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-podinfo\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.622115 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.622091    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-token\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.622355 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.622340    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfl48\" (UniqueName: \"kubernetes.io/projected/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-kube-api-access-qfl48\") pod \"e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") " pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.811549 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.811457    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:27.985179 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.985149    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"]
Jul 24 16:21:27.986727 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:21:27.986696    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91440df6_cce1_4af2_a2bf_b2243f0d5fc2.slice/crio-b9e1f94c9a1a9037c434557d8bd6a3759b3f921a554c83f034695eba5ef75f36 WatchSource:0}: Error finding container b9e1f94c9a1a9037c434557d8bd6a3759b3f921a554c83f034695eba5ef75f36: Status 404 returned error can't find the container with id b9e1f94c9a1a9037c434557d8bd6a3759b3f921a554c83f034695eba5ef75f36
Jul 24 16:21:27.988852 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.988823    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:21:27.988913 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.988886    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:21:27.988913 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:27.988914    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:21:28.594383 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:28.594342    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh" event={"ID":"91440df6-cce1-4af2-a2bf-b2243f0d5fc2","Type":"ContainerStarted","Data":"2638368cc0d47daa2ebb5ec2a030952ede51510b0d680696d0ce97c02938af00"}
Jul 24 16:21:28.594873 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:28.594421    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh" event={"ID":"91440df6-cce1-4af2-a2bf-b2243f0d5fc2","Type":"ContainerStarted","Data":"b9e1f94c9a1a9037c434557d8bd6a3759b3f921a554c83f034695eba5ef75f36"}
Jul 24 16:21:28.618303 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:28.618238    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh" podStartSLOduration=1.618221998 podStartE2EDuration="1.618221998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:21:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:21:28.615594299 +0000 UTC m=+2167.899921928" watchObservedRunningTime="2026-07-24 16:21:28.618221998 +0000 UTC m=+2167.902549604"
Jul 24 16:21:28.811803 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:28.811770    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:29.817607 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:29.817577    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:30.603417 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:30.603385    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:30.604487 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:30.604465    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:35.909535 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:35.909498    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"]
Jul 24 16:21:35.913290 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:35.913266    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:35.918077 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:35.918051    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-573c9b-openshift-default-dockercfg-vhj8k\""
Jul 24 16:21:35.934511 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:35.934485    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"]
Jul 24 16:21:35.991366 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:35.991314    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-credential-socket\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:35.991366 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:35.991366    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-podinfo\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:35.991578 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:35.991401    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istiod-ca-cert\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:35.991578 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:35.991475    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-workload-socket\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:35.991578 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:35.991520    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v47ll\" (UniqueName: \"kubernetes.io/projected/a3f986a0-8867-4899-8d41-fb3f6dcd3822-kube-api-access-v47ll\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:35.991706 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:35.991585    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-envoy\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:35.991706 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:35.991611    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-workload-certs\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:35.991706 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:35.991682    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-data\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:35.991826 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:35.991723    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-token\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.092218 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.092178    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-envoy\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.092218 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.092219    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-workload-certs\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.092476 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.092235    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-data\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.092476 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.092254    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-token\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.092476 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.092298    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-credential-socket\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.092476 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.092438    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-podinfo\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.092690 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.092490    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istiod-ca-cert\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.092690 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.092590    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-workload-socket\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.092690 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.092652    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v47ll\" (UniqueName: \"kubernetes.io/projected/a3f986a0-8867-4899-8d41-fb3f6dcd3822-kube-api-access-v47ll\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.092690 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.092672    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-workload-certs\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.092891 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.092794    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-data\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.092994 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.092941    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-workload-socket\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.093102 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.093086    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-credential-socket\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.093455 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.093432    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istiod-ca-cert\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.095031 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.095002    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-podinfo\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.095156 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.095044    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-envoy\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.102272 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.102249    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-token\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.103583 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.103563    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v47ll\" (UniqueName: \"kubernetes.io/projected/a3f986a0-8867-4899-8d41-fb3f6dcd3822-kube-api-access-v47ll\") pod \"e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") " pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.224475 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.224374    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:36.387396 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.387370    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"]
Jul 24 16:21:36.391363 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:21:36.391332    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3f986a0_8867_4899_8d41_fb3f6dcd3822.slice/crio-2bb4b6fb2d70656742806a23c948b982427cd6c60556d8e79fada83d6d36d09a WatchSource:0}: Error finding container 2bb4b6fb2d70656742806a23c948b982427cd6c60556d8e79fada83d6d36d09a: Status 404 returned error can't find the container with id 2bb4b6fb2d70656742806a23c948b982427cd6c60556d8e79fada83d6d36d09a
Jul 24 16:21:36.393408 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.393375    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:21:36.393525 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.393455    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:21:36.393525 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.393498    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:21:36.631768 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.631679    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2" event={"ID":"a3f986a0-8867-4899-8d41-fb3f6dcd3822","Type":"ContainerStarted","Data":"e6a1db6c3f42672cb48d38dca9f90f8f73b33a2d107db90cd415d9b4fc540b92"}
Jul 24 16:21:36.631768 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.631741    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2" event={"ID":"a3f986a0-8867-4899-8d41-fb3f6dcd3822","Type":"ContainerStarted","Data":"2bb4b6fb2d70656742806a23c948b982427cd6c60556d8e79fada83d6d36d09a"}
Jul 24 16:21:36.656207 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:36.656150    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2" podStartSLOduration=1.65613113 podStartE2EDuration="1.65613113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:21:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:21:36.654517041 +0000 UTC m=+2175.938844649" watchObservedRunningTime="2026-07-24 16:21:36.65613113 +0000 UTC m=+2175.940458738"
Jul 24 16:21:37.225524 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:37.225495    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:37.231100 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:37.231074    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:37.636404 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:37.636298    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:37.637275 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:37.637250    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:45.844315 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:45.843471    2572 patch_prober.go:28] interesting pod/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": read tcp 10.132.0.2:57268->10.132.0.39:15021: read: connection reset by peer" start-of-body=
Jul 24 16:21:45.844315 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:45.843536    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh" podUID="91440df6-cce1-4af2-a2bf-b2243f0d5fc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": read tcp 10.132.0.2:57268->10.132.0.39:15021: read: connection reset by peer"
Jul 24 16:21:46.678335 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:46.678303    2572 generic.go:356] "Generic (PLEG): container finished" podID="91440df6-cce1-4af2-a2bf-b2243f0d5fc2" containerID="2638368cc0d47daa2ebb5ec2a030952ede51510b0d680696d0ce97c02938af00" exitCode=0
Jul 24 16:21:46.678515 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:46.678375    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh" event={"ID":"91440df6-cce1-4af2-a2bf-b2243f0d5fc2","Type":"ContainerDied","Data":"2638368cc0d47daa2ebb5ec2a030952ede51510b0d680696d0ce97c02938af00"}
Jul 24 16:21:46.678800 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:46.678788    2572 scope.go:117] "RemoveContainer" containerID="2638368cc0d47daa2ebb5ec2a030952ede51510b0d680696d0ce97c02938af00"
Jul 24 16:21:46.679816 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:46.679789    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:21:46.679874 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:46.679856    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:21:46.679909 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:46.679891    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:21:47.684598 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:47.684557    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh" event={"ID":"91440df6-cce1-4af2-a2bf-b2243f0d5fc2","Type":"ContainerStarted","Data":"750fa7931be994c6976be0e0fc2ed92ae7116c4e2ccfa85318864d3dcf493d13"}
Jul 24 16:21:47.812437 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:47.812374    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:47.813001 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:47.812970    2572 patch_prober.go:28] interesting pod/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 24 16:21:47.813127 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:47.813045    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh" podUID="91440df6-cce1-4af2-a2bf-b2243f0d5fc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 24 16:21:48.812367 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:48.812331    2572 patch_prober.go:28] interesting pod/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 24 16:21:48.814102 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:48.814067    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh" podUID="91440df6-cce1-4af2-a2bf-b2243f0d5fc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 24 16:21:49.814543 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:49.814501    2572 patch_prober.go:28] interesting pod/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 24 16:21:49.815074 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:49.814585    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh" podUID="91440df6-cce1-4af2-a2bf-b2243f0d5fc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 24 16:21:50.813033 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:50.812990    2572 patch_prober.go:28] interesting pod/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body=
Jul 24 16:21:50.813834 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:50.813332    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh" podUID="91440df6-cce1-4af2-a2bf-b2243f0d5fc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused"
Jul 24 16:21:51.815384 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:51.815352    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:51.815835 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:51.815634    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:51.816429 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:51.816414    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:21:52.638368 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:52.638334    2572 patch_prober.go:28] interesting pod/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 24 16:21:52.638572 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:52.638403    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2" podUID="a3f986a0-8867-4899-8d41-fb3f6dcd3822" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 24 16:21:52.708790 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:52.708711    2572 generic.go:356] "Generic (PLEG): container finished" podID="a3f986a0-8867-4899-8d41-fb3f6dcd3822" containerID="e6a1db6c3f42672cb48d38dca9f90f8f73b33a2d107db90cd415d9b4fc540b92" exitCode=0
Jul 24 16:21:52.708960 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:52.708784    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2" event={"ID":"a3f986a0-8867-4899-8d41-fb3f6dcd3822","Type":"ContainerDied","Data":"e6a1db6c3f42672cb48d38dca9f90f8f73b33a2d107db90cd415d9b4fc540b92"}
Jul 24 16:21:52.709317 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:52.709299    2572 scope.go:117] "RemoveContainer" containerID="e6a1db6c3f42672cb48d38dca9f90f8f73b33a2d107db90cd415d9b4fc540b92"
Jul 24 16:21:52.710281 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:52.710258    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:21:52.710359 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:52.710326    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:21:52.710420 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:52.710373    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:21:53.726402 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:53.726365    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2" event={"ID":"a3f986a0-8867-4899-8d41-fb3f6dcd3822","Type":"ContainerStarted","Data":"d4e16e6f365b21c2be863de6194c6019daa9e120e2823d483fbe9235709081b3"}
Jul 24 16:21:54.224614 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:54.224574    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:54.224815 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:54.224764    2572 patch_prober.go:28] interesting pod/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 24 16:21:54.224882 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:54.224827    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2" podUID="a3f986a0-8867-4899-8d41-fb3f6dcd3822" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 24 16:21:55.224888 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:55.224852    2572 patch_prober.go:28] interesting pod/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 24 16:21:55.225317 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:55.224958    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2" podUID="a3f986a0-8867-4899-8d41-fb3f6dcd3822" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 24 16:21:56.225063 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:56.225025    2572 patch_prober.go:28] interesting pod/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 24 16:21:56.225573 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:56.225085    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2" podUID="a3f986a0-8867-4899-8d41-fb3f6dcd3822" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 24 16:21:57.224938 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:57.224893    2572 patch_prober.go:28] interesting pod/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 24 16:21:57.225119 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:57.224975    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2" podUID="a3f986a0-8867-4899-8d41-fb3f6dcd3822" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 24 16:21:58.228935 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:58.228882    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:58.229425 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:58.229408    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:58.230295 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:58.230270    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:21:58.987308 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:58.987269    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"]
Jul 24 16:21:58.989747 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:58.989729    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:58.993509 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:58.993484    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-57d2c2\"/\"test-model-57d2c2-kserve-self-signed-certs\""
Jul 24 16:21:58.993650 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:58.993511    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-57d2c2\"/\"kube-root-ca.crt\""
Jul 24 16:21:58.993650 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:58.993492    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-57d2c2\"/\"openshift-service-ca.crt\""
Jul 24 16:21:58.993650 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:58.993610    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-57d2c2\"/\"default-dockercfg-k7h5r\""
Jul 24 16:21:58.998835 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:58.998813    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"]
Jul 24 16:21:59.120720 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.120674    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-dshm\") pod \"test-model-57d2c2-kserve-6dfd46989f-8spc2\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") " pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.120720 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.120716    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-tls-certs\") pod \"test-model-57d2c2-kserve-6dfd46989f-8spc2\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") " pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.121008 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.120762    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k58bn\" (UniqueName: \"kubernetes.io/projected/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-kube-api-access-k58bn\") pod \"test-model-57d2c2-kserve-6dfd46989f-8spc2\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") " pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.121008 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.120781    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-tmp-dir\") pod \"test-model-57d2c2-kserve-6dfd46989f-8spc2\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") " pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.121008 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.120811    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-model-cache\") pod \"test-model-57d2c2-kserve-6dfd46989f-8spc2\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") " pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.121008 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.120841    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-home\") pod \"test-model-57d2c2-kserve-6dfd46989f-8spc2\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") " pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.221518 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.221480    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-dshm\") pod \"test-model-57d2c2-kserve-6dfd46989f-8spc2\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") " pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.221722 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.221528    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-tls-certs\") pod \"test-model-57d2c2-kserve-6dfd46989f-8spc2\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") " pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.221722 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.221567    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k58bn\" (UniqueName: \"kubernetes.io/projected/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-kube-api-access-k58bn\") pod \"test-model-57d2c2-kserve-6dfd46989f-8spc2\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") " pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.221722 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.221591    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-tmp-dir\") pod \"test-model-57d2c2-kserve-6dfd46989f-8spc2\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") " pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.221886 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.221744    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-model-cache\") pod \"test-model-57d2c2-kserve-6dfd46989f-8spc2\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") " pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.221886 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.221791    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-home\") pod \"test-model-57d2c2-kserve-6dfd46989f-8spc2\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") " pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.222013 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.221985    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-tmp-dir\") pod \"test-model-57d2c2-kserve-6dfd46989f-8spc2\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") " pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.222248 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.222228    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-home\") pod \"test-model-57d2c2-kserve-6dfd46989f-8spc2\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") " pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.222400 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.222379    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-model-cache\") pod \"test-model-57d2c2-kserve-6dfd46989f-8spc2\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") " pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.223998 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.223977    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-dshm\") pod \"test-model-57d2c2-kserve-6dfd46989f-8spc2\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") " pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.224605 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.224588    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-tls-certs\") pod \"test-model-57d2c2-kserve-6dfd46989f-8spc2\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") " pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.230146 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.230123    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k58bn\" (UniqueName: \"kubernetes.io/projected/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-kube-api-access-k58bn\") pod \"test-model-57d2c2-kserve-6dfd46989f-8spc2\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") " pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.300216 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.300100    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:21:59.435580 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.435551    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"]
Jul 24 16:21:59.437760 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:21:59.437725    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6879b4d_f68a_413b_9d2f_bd73e1b1b698.slice/crio-e8e2ba83c53bf56129b4d676bf6e9e50fd145ebf013d6b2bc722027c5971a66a WatchSource:0}: Error finding container e8e2ba83c53bf56129b4d676bf6e9e50fd145ebf013d6b2bc722027c5971a66a: Status 404 returned error can't find the container with id e8e2ba83c53bf56129b4d676bf6e9e50fd145ebf013d6b2bc722027c5971a66a
Jul 24 16:21:59.753902 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.753859    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2" event={"ID":"e6879b4d-f68a-413b-9d2f-bd73e1b1b698","Type":"ContainerStarted","Data":"6f7c596210be3e2ee7fb7afbda5fafcaed3e6919efc59c28b2496f0dd951c4b1"}
Jul 24 16:21:59.754110 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.753950    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2" event={"ID":"e6879b4d-f68a-413b-9d2f-bd73e1b1b698","Type":"ContainerStarted","Data":"e8e2ba83c53bf56129b4d676bf6e9e50fd145ebf013d6b2bc722027c5971a66a"}
Jul 24 16:21:59.775750 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:21:59.775698    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2" podStartSLOduration=1.775677924 podStartE2EDuration="1.775677924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:21:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:21:59.774171321 +0000 UTC m=+2199.058498928" watchObservedRunningTime="2026-07-24 16:21:59.775677924 +0000 UTC m=+2199.060005530"
Jul 24 16:22:09.301022 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:09.300979    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:22:09.301022 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:09.301032    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:22:09.313621 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:09.313594    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:22:09.808647 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:09.808621    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:22:42.289871 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.289794    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"]
Jul 24 16:22:42.293481 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.293454    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.296983 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.296956    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-573c9b\"/\"default-dockercfg-btmzm\""
Jul 24 16:22:42.297118 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.296991    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-573c9b\"/\"openshift-service-ca.crt\""
Jul 24 16:22:42.297118 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.297014    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-573c9b\"/\"kube-root-ca.crt\""
Jul 24 16:22:42.297118 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.296996    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-573c9b\"/\"test-model-573c9b-kserve-self-signed-certs\""
Jul 24 16:22:42.306369 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.306348    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"]
Jul 24 16:22:42.420221 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.420183    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhlqm\" (UniqueName: \"kubernetes.io/projected/f3caafff-d976-43c5-97ce-a1dfdaf0a459-kube-api-access-qhlqm\") pod \"test-model-573c9b-kserve-7dd6bd6bdc-2l47k\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") " pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.420426 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.420260    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-tmp-dir\") pod \"test-model-573c9b-kserve-7dd6bd6bdc-2l47k\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") " pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.420426 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.420292    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-model-cache\") pod \"test-model-573c9b-kserve-7dd6bd6bdc-2l47k\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") " pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.420426 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.420381    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3caafff-d976-43c5-97ce-a1dfdaf0a459-tls-certs\") pod \"test-model-573c9b-kserve-7dd6bd6bdc-2l47k\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") " pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.420602 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.420445    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-home\") pod \"test-model-573c9b-kserve-7dd6bd6bdc-2l47k\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") " pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.420602 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.420498    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-dshm\") pod \"test-model-573c9b-kserve-7dd6bd6bdc-2l47k\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") " pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.521605 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.521569    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-tmp-dir\") pod \"test-model-573c9b-kserve-7dd6bd6bdc-2l47k\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") " pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.521759 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.521619    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-model-cache\") pod \"test-model-573c9b-kserve-7dd6bd6bdc-2l47k\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") " pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.521759 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.521660    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3caafff-d976-43c5-97ce-a1dfdaf0a459-tls-certs\") pod \"test-model-573c9b-kserve-7dd6bd6bdc-2l47k\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") " pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.521759 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.521691    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-home\") pod \"test-model-573c9b-kserve-7dd6bd6bdc-2l47k\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") " pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.521759 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.521723    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-dshm\") pod \"test-model-573c9b-kserve-7dd6bd6bdc-2l47k\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") " pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.521759 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.521746    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhlqm\" (UniqueName: \"kubernetes.io/projected/f3caafff-d976-43c5-97ce-a1dfdaf0a459-kube-api-access-qhlqm\") pod \"test-model-573c9b-kserve-7dd6bd6bdc-2l47k\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") " pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.522163 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.522133    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-tmp-dir\") pod \"test-model-573c9b-kserve-7dd6bd6bdc-2l47k\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") " pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.522235 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.522173    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-home\") pod \"test-model-573c9b-kserve-7dd6bd6bdc-2l47k\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") " pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.522235 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.522183    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-model-cache\") pod \"test-model-573c9b-kserve-7dd6bd6bdc-2l47k\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") " pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.524254 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.524232    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-dshm\") pod \"test-model-573c9b-kserve-7dd6bd6bdc-2l47k\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") " pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.524506 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.524491    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3caafff-d976-43c5-97ce-a1dfdaf0a459-tls-certs\") pod \"test-model-573c9b-kserve-7dd6bd6bdc-2l47k\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") " pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.530343 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.530320    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhlqm\" (UniqueName: \"kubernetes.io/projected/f3caafff-d976-43c5-97ce-a1dfdaf0a459-kube-api-access-qhlqm\") pod \"test-model-573c9b-kserve-7dd6bd6bdc-2l47k\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") " pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.604833 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.604728    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:42.737292 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.737260    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"]
Jul 24 16:22:42.740678 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:22:42.740649    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3caafff_d976_43c5_97ce_a1dfdaf0a459.slice/crio-6bc464c15d60df7cbd3d622daefe0850927ba54d962e8e1a3778667babe7cc63 WatchSource:0}: Error finding container 6bc464c15d60df7cbd3d622daefe0850927ba54d962e8e1a3778667babe7cc63: Status 404 returned error can't find the container with id 6bc464c15d60df7cbd3d622daefe0850927ba54d962e8e1a3778667babe7cc63
Jul 24 16:22:42.951076 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.951042    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k" event={"ID":"f3caafff-d976-43c5-97ce-a1dfdaf0a459","Type":"ContainerStarted","Data":"6bac73a8b193dcc43846bae10ed08e817f970cdfe066de498fc21d0ed18655d5"}
Jul 24 16:22:42.951076 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.951087    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k" event={"ID":"f3caafff-d976-43c5-97ce-a1dfdaf0a459","Type":"ContainerStarted","Data":"6bc464c15d60df7cbd3d622daefe0850927ba54d962e8e1a3778667babe7cc63"}
Jul 24 16:22:42.969803 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:42.969741    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k" podStartSLOduration=0.969726934 podStartE2EDuration="969.726934ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:22:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:22:42.967553834 +0000 UTC m=+2242.251881453" watchObservedRunningTime="2026-07-24 16:22:42.969726934 +0000 UTC m=+2242.254054541"
Jul 24 16:22:52.605082 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:52.605043    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:52.605647 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:52.605131    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:52.617781 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:52.617754    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:22:53.006597 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:22:53.006570    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:23:33.828615 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:33.828580    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"]
Jul 24 16:23:34.888510 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:34.888469    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"]
Jul 24 16:23:48.847977 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:48.847884    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2" podUID="e6879b4d-f68a-413b-9d2f-bd73e1b1b698" containerName="main" containerID="cri-o://6f7c596210be3e2ee7fb7afbda5fafcaed3e6919efc59c28b2496f0dd951c4b1" gracePeriod=45
Jul 24 16:23:49.103581 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.103513    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:23:49.138091 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.138058    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-dshm\") pod \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") "
Jul 24 16:23:49.138289 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.138100    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-tls-certs\") pod \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") "
Jul 24 16:23:49.138289 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.138138    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-home\") pod \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") "
Jul 24 16:23:49.138289 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.138205    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k58bn\" (UniqueName: \"kubernetes.io/projected/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-kube-api-access-k58bn\") pod \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") "
Jul 24 16:23:49.138289 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.138234    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-model-cache\") pod \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") "
Jul 24 16:23:49.138289 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.138258    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-tmp-dir\") pod \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\" (UID: \"e6879b4d-f68a-413b-9d2f-bd73e1b1b698\") "
Jul 24 16:23:49.138546 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.138447    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-home" (OuterVolumeSpecName: "home") pod "e6879b4d-f68a-413b-9d2f-bd73e1b1b698" (UID: "e6879b4d-f68a-413b-9d2f-bd73e1b1b698"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:23:49.138546 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.138505    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-model-cache" (OuterVolumeSpecName: "model-cache") pod "e6879b4d-f68a-413b-9d2f-bd73e1b1b698" (UID: "e6879b4d-f68a-413b-9d2f-bd73e1b1b698"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:23:49.138664 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.138597    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-home\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:23:49.138664 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.138610    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-model-cache\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:23:49.138750 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.138678    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e6879b4d-f68a-413b-9d2f-bd73e1b1b698" (UID: "e6879b4d-f68a-413b-9d2f-bd73e1b1b698"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:23:49.140470 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.140441    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-dshm" (OuterVolumeSpecName: "dshm") pod "e6879b4d-f68a-413b-9d2f-bd73e1b1b698" (UID: "e6879b4d-f68a-413b-9d2f-bd73e1b1b698"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:23:49.140592 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.140532    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-kube-api-access-k58bn" (OuterVolumeSpecName: "kube-api-access-k58bn") pod "e6879b4d-f68a-413b-9d2f-bd73e1b1b698" (UID: "e6879b4d-f68a-413b-9d2f-bd73e1b1b698"). InnerVolumeSpecName "kube-api-access-k58bn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:23:49.140784 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.140760    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e6879b4d-f68a-413b-9d2f-bd73e1b1b698" (UID: "e6879b4d-f68a-413b-9d2f-bd73e1b1b698"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:23:49.239600 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.239570    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-dshm\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:23:49.239600 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.239599    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-tls-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:23:49.239806 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.239611    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k58bn\" (UniqueName: \"kubernetes.io/projected/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-kube-api-access-k58bn\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:23:49.239806 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.239620    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6879b4d-f68a-413b-9d2f-bd73e1b1b698-tmp-dir\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:23:49.245866 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.245840    2572 generic.go:356] "Generic (PLEG): container finished" podID="e6879b4d-f68a-413b-9d2f-bd73e1b1b698" containerID="6f7c596210be3e2ee7fb7afbda5fafcaed3e6919efc59c28b2496f0dd951c4b1" exitCode=0
Jul 24 16:23:49.246044 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.245883    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"
Jul 24 16:23:49.246044 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.245885    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2" event={"ID":"e6879b4d-f68a-413b-9d2f-bd73e1b1b698","Type":"ContainerDied","Data":"6f7c596210be3e2ee7fb7afbda5fafcaed3e6919efc59c28b2496f0dd951c4b1"}
Jul 24 16:23:49.246044 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.245962    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2" event={"ID":"e6879b4d-f68a-413b-9d2f-bd73e1b1b698","Type":"ContainerDied","Data":"e8e2ba83c53bf56129b4d676bf6e9e50fd145ebf013d6b2bc722027c5971a66a"}
Jul 24 16:23:49.246044 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.245982    2572 scope.go:117] "RemoveContainer" containerID="6f7c596210be3e2ee7fb7afbda5fafcaed3e6919efc59c28b2496f0dd951c4b1"
Jul 24 16:23:49.254336 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.254313    2572 scope.go:117] "RemoveContainer" containerID="6f7c596210be3e2ee7fb7afbda5fafcaed3e6919efc59c28b2496f0dd951c4b1"
Jul 24 16:23:49.254606 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:23:49.254588    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f7c596210be3e2ee7fb7afbda5fafcaed3e6919efc59c28b2496f0dd951c4b1\": container with ID starting with 6f7c596210be3e2ee7fb7afbda5fafcaed3e6919efc59c28b2496f0dd951c4b1 not found: ID does not exist" containerID="6f7c596210be3e2ee7fb7afbda5fafcaed3e6919efc59c28b2496f0dd951c4b1"
Jul 24 16:23:49.254676 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.254614    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f7c596210be3e2ee7fb7afbda5fafcaed3e6919efc59c28b2496f0dd951c4b1"} err="failed to get container status \"6f7c596210be3e2ee7fb7afbda5fafcaed3e6919efc59c28b2496f0dd951c4b1\": rpc error: code = NotFound desc = could not find container \"6f7c596210be3e2ee7fb7afbda5fafcaed3e6919efc59c28b2496f0dd951c4b1\": container with ID starting with 6f7c596210be3e2ee7fb7afbda5fafcaed3e6919efc59c28b2496f0dd951c4b1 not found: ID does not exist"
Jul 24 16:23:49.263876 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.263848    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"]
Jul 24 16:23:49.267666 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.267635    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-57d2c2/test-model-57d2c2-kserve-6dfd46989f-8spc2"]
Jul 24 16:23:49.908872 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:49.908824    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k" podUID="f3caafff-d976-43c5-97ce-a1dfdaf0a459" containerName="main" containerID="cri-o://6bac73a8b193dcc43846bae10ed08e817f970cdfe066de498fc21d0ed18655d5" gracePeriod=45
Jul 24 16:23:50.158246 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.158218    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:23:50.248200 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.248116    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-tmp-dir\") pod \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") "
Jul 24 16:23:50.248200 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.248179    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3caafff-d976-43c5-97ce-a1dfdaf0a459-tls-certs\") pod \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") "
Jul 24 16:23:50.248440 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.248254    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-dshm\") pod \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") "
Jul 24 16:23:50.248440 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.248291    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhlqm\" (UniqueName: \"kubernetes.io/projected/f3caafff-d976-43c5-97ce-a1dfdaf0a459-kube-api-access-qhlqm\") pod \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") "
Jul 24 16:23:50.248440 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.248354    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-home\") pod \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") "
Jul 24 16:23:50.248440 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.248396    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-model-cache\") pod \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\" (UID: \"f3caafff-d976-43c5-97ce-a1dfdaf0a459\") "
Jul 24 16:23:50.248440 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.248429    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f3caafff-d976-43c5-97ce-a1dfdaf0a459" (UID: "f3caafff-d976-43c5-97ce-a1dfdaf0a459"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:23:50.248697 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.248636    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-home" (OuterVolumeSpecName: "home") pod "f3caafff-d976-43c5-97ce-a1dfdaf0a459" (UID: "f3caafff-d976-43c5-97ce-a1dfdaf0a459"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:23:50.248749 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.248703    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-model-cache" (OuterVolumeSpecName: "model-cache") pod "f3caafff-d976-43c5-97ce-a1dfdaf0a459" (UID: "f3caafff-d976-43c5-97ce-a1dfdaf0a459"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:23:50.248749 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.248725    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-tmp-dir\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:23:50.248749 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.248744    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-home\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:23:50.250671 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.250640    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3caafff-d976-43c5-97ce-a1dfdaf0a459-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f3caafff-d976-43c5-97ce-a1dfdaf0a459" (UID: "f3caafff-d976-43c5-97ce-a1dfdaf0a459"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:23:50.250792 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.250747    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3caafff-d976-43c5-97ce-a1dfdaf0a459-kube-api-access-qhlqm" (OuterVolumeSpecName: "kube-api-access-qhlqm") pod "f3caafff-d976-43c5-97ce-a1dfdaf0a459" (UID: "f3caafff-d976-43c5-97ce-a1dfdaf0a459"). InnerVolumeSpecName "kube-api-access-qhlqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:23:50.250792 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.250755    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-dshm" (OuterVolumeSpecName: "dshm") pod "f3caafff-d976-43c5-97ce-a1dfdaf0a459" (UID: "f3caafff-d976-43c5-97ce-a1dfdaf0a459"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:23:50.253284 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.253255    2572 generic.go:356] "Generic (PLEG): container finished" podID="f3caafff-d976-43c5-97ce-a1dfdaf0a459" containerID="6bac73a8b193dcc43846bae10ed08e817f970cdfe066de498fc21d0ed18655d5" exitCode=0
Jul 24 16:23:50.253378 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.253301    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"
Jul 24 16:23:50.253378 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.253337    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k" event={"ID":"f3caafff-d976-43c5-97ce-a1dfdaf0a459","Type":"ContainerDied","Data":"6bac73a8b193dcc43846bae10ed08e817f970cdfe066de498fc21d0ed18655d5"}
Jul 24 16:23:50.253459 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.253384    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k" event={"ID":"f3caafff-d976-43c5-97ce-a1dfdaf0a459","Type":"ContainerDied","Data":"6bc464c15d60df7cbd3d622daefe0850927ba54d962e8e1a3778667babe7cc63"}
Jul 24 16:23:50.253459 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.253401    2572 scope.go:117] "RemoveContainer" containerID="6bac73a8b193dcc43846bae10ed08e817f970cdfe066de498fc21d0ed18655d5"
Jul 24 16:23:50.268209 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.268188    2572 scope.go:117] "RemoveContainer" containerID="6bac73a8b193dcc43846bae10ed08e817f970cdfe066de498fc21d0ed18655d5"
Jul 24 16:23:50.268515 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:23:50.268498    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bac73a8b193dcc43846bae10ed08e817f970cdfe066de498fc21d0ed18655d5\": container with ID starting with 6bac73a8b193dcc43846bae10ed08e817f970cdfe066de498fc21d0ed18655d5 not found: ID does not exist" containerID="6bac73a8b193dcc43846bae10ed08e817f970cdfe066de498fc21d0ed18655d5"
Jul 24 16:23:50.268575 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.268525    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bac73a8b193dcc43846bae10ed08e817f970cdfe066de498fc21d0ed18655d5"} err="failed to get container status \"6bac73a8b193dcc43846bae10ed08e817f970cdfe066de498fc21d0ed18655d5\": rpc error: code = NotFound desc = could not find container \"6bac73a8b193dcc43846bae10ed08e817f970cdfe066de498fc21d0ed18655d5\": container with ID starting with 6bac73a8b193dcc43846bae10ed08e817f970cdfe066de498fc21d0ed18655d5 not found: ID does not exist"
Jul 24 16:23:50.279725 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.279699    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"]
Jul 24 16:23:50.283891 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.283866    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-573c9b/test-model-573c9b-kserve-7dd6bd6bdc-2l47k"]
Jul 24 16:23:50.349509 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.349472    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhlqm\" (UniqueName: \"kubernetes.io/projected/f3caafff-d976-43c5-97ce-a1dfdaf0a459-kube-api-access-qhlqm\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:23:50.349509 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.349513    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-model-cache\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:23:50.349783 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.349530    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3caafff-d976-43c5-97ce-a1dfdaf0a459-tls-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:23:50.349783 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:50.349543    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3caafff-d976-43c5-97ce-a1dfdaf0a459-dshm\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:23:51.223360 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:51.223325    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6879b4d-f68a-413b-9d2f-bd73e1b1b698" path="/var/lib/kubelet/pods/e6879b4d-f68a-413b-9d2f-bd73e1b1b698/volumes"
Jul 24 16:23:51.223768 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:51.223713    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3caafff-d976-43c5-97ce-a1dfdaf0a459" path="/var/lib/kubelet/pods/f3caafff-d976-43c5-97ce-a1dfdaf0a459/volumes"
Jul 24 16:23:56.881582 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:56.881548    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"]
Jul 24 16:23:56.882047 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:23:56.881790    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh" podUID="91440df6-cce1-4af2-a2bf-b2243f0d5fc2" containerName="istio-proxy" containerID="cri-o://750fa7931be994c6976be0e0fc2ed92ae7116c4e2ccfa85318864d3dcf493d13" gracePeriod=30
Jul 24 16:24:02.134616 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.134587    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:24:02.256146 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.256055    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-workload-socket\") pod \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") "
Jul 24 16:24:02.256146 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.256096    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-token\") pod \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") "
Jul 24 16:24:02.256146 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.256130    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-credential-socket\") pod \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") "
Jul 24 16:24:02.256439 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.256158    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-data\") pod \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") "
Jul 24 16:24:02.256439 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.256178    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-podinfo\") pod \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") "
Jul 24 16:24:02.256439 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.256199    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfl48\" (UniqueName: \"kubernetes.io/projected/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-kube-api-access-qfl48\") pod \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") "
Jul 24 16:24:02.256439 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.256270    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-envoy\") pod \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") "
Jul 24 16:24:02.256439 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.256294    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-workload-certs\") pod \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") "
Jul 24 16:24:02.256439 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.256325    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istiod-ca-cert\") pod \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\" (UID: \"91440df6-cce1-4af2-a2bf-b2243f0d5fc2\") "
Jul 24 16:24:02.256439 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.256403    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "91440df6-cce1-4af2-a2bf-b2243f0d5fc2" (UID: "91440df6-cce1-4af2-a2bf-b2243f0d5fc2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:24:02.256439 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.256418    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "91440df6-cce1-4af2-a2bf-b2243f0d5fc2" (UID: "91440df6-cce1-4af2-a2bf-b2243f0d5fc2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:24:02.256962 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.256637    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-workload-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:24:02.256962 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.256643    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "91440df6-cce1-4af2-a2bf-b2243f0d5fc2" (UID: "91440df6-cce1-4af2-a2bf-b2243f0d5fc2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:24:02.256962 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.256659    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-credential-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:24:02.256962 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.256747    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "91440df6-cce1-4af2-a2bf-b2243f0d5fc2" (UID: "91440df6-cce1-4af2-a2bf-b2243f0d5fc2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:24:02.257352 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.257075    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-data" (OuterVolumeSpecName: "istio-data") pod "91440df6-cce1-4af2-a2bf-b2243f0d5fc2" (UID: "91440df6-cce1-4af2-a2bf-b2243f0d5fc2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:24:02.258820 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.258784    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "91440df6-cce1-4af2-a2bf-b2243f0d5fc2" (UID: "91440df6-cce1-4af2-a2bf-b2243f0d5fc2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:24:02.258820 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.258795    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-kube-api-access-qfl48" (OuterVolumeSpecName: "kube-api-access-qfl48") pod "91440df6-cce1-4af2-a2bf-b2243f0d5fc2" (UID: "91440df6-cce1-4af2-a2bf-b2243f0d5fc2"). InnerVolumeSpecName "kube-api-access-qfl48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:24:02.259216 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.259188    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "91440df6-cce1-4af2-a2bf-b2243f0d5fc2" (UID: "91440df6-cce1-4af2-a2bf-b2243f0d5fc2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 16:24:02.259310 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.259219    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-token" (OuterVolumeSpecName: "istio-token") pod "91440df6-cce1-4af2-a2bf-b2243f0d5fc2" (UID: "91440df6-cce1-4af2-a2bf-b2243f0d5fc2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:24:02.310860 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.310819    2572 generic.go:356] "Generic (PLEG): container finished" podID="91440df6-cce1-4af2-a2bf-b2243f0d5fc2" containerID="750fa7931be994c6976be0e0fc2ed92ae7116c4e2ccfa85318864d3dcf493d13" exitCode=0
Jul 24 16:24:02.311064 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.310866    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"
Jul 24 16:24:02.311064 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.310907    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh" event={"ID":"91440df6-cce1-4af2-a2bf-b2243f0d5fc2","Type":"ContainerDied","Data":"750fa7931be994c6976be0e0fc2ed92ae7116c4e2ccfa85318864d3dcf493d13"}
Jul 24 16:24:02.311064 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.310990    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh" event={"ID":"91440df6-cce1-4af2-a2bf-b2243f0d5fc2","Type":"ContainerDied","Data":"b9e1f94c9a1a9037c434557d8bd6a3759b3f921a554c83f034695eba5ef75f36"}
Jul 24 16:24:02.311064 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.311012    2572 scope.go:117] "RemoveContainer" containerID="750fa7931be994c6976be0e0fc2ed92ae7116c4e2ccfa85318864d3dcf493d13"
Jul 24 16:24:02.320452 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.320435    2572 scope.go:117] "RemoveContainer" containerID="2638368cc0d47daa2ebb5ec2a030952ede51510b0d680696d0ce97c02938af00"
Jul 24 16:24:02.329758 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.329740    2572 scope.go:117] "RemoveContainer" containerID="750fa7931be994c6976be0e0fc2ed92ae7116c4e2ccfa85318864d3dcf493d13"
Jul 24 16:24:02.330152 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:24:02.330124    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"750fa7931be994c6976be0e0fc2ed92ae7116c4e2ccfa85318864d3dcf493d13\": container with ID starting with 750fa7931be994c6976be0e0fc2ed92ae7116c4e2ccfa85318864d3dcf493d13 not found: ID does not exist" containerID="750fa7931be994c6976be0e0fc2ed92ae7116c4e2ccfa85318864d3dcf493d13"
Jul 24 16:24:02.330228 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.330166    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"750fa7931be994c6976be0e0fc2ed92ae7116c4e2ccfa85318864d3dcf493d13"} err="failed to get container status \"750fa7931be994c6976be0e0fc2ed92ae7116c4e2ccfa85318864d3dcf493d13\": rpc error: code = NotFound desc = could not find container \"750fa7931be994c6976be0e0fc2ed92ae7116c4e2ccfa85318864d3dcf493d13\": container with ID starting with 750fa7931be994c6976be0e0fc2ed92ae7116c4e2ccfa85318864d3dcf493d13 not found: ID does not exist"
Jul 24 16:24:02.330228 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.330192    2572 scope.go:117] "RemoveContainer" containerID="2638368cc0d47daa2ebb5ec2a030952ede51510b0d680696d0ce97c02938af00"
Jul 24 16:24:02.330480 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:24:02.330457    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2638368cc0d47daa2ebb5ec2a030952ede51510b0d680696d0ce97c02938af00\": container with ID starting with 2638368cc0d47daa2ebb5ec2a030952ede51510b0d680696d0ce97c02938af00 not found: ID does not exist" containerID="2638368cc0d47daa2ebb5ec2a030952ede51510b0d680696d0ce97c02938af00"
Jul 24 16:24:02.330578 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.330481    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2638368cc0d47daa2ebb5ec2a030952ede51510b0d680696d0ce97c02938af00"} err="failed to get container status \"2638368cc0d47daa2ebb5ec2a030952ede51510b0d680696d0ce97c02938af00\": rpc error: code = NotFound desc = could not find container \"2638368cc0d47daa2ebb5ec2a030952ede51510b0d680696d0ce97c02938af00\": container with ID starting with 2638368cc0d47daa2ebb5ec2a030952ede51510b0d680696d0ce97c02938af00 not found: ID does not exist"
Jul 24 16:24:02.341324 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.341298    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"]
Jul 24 16:24:02.357469 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.357435    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-57d2c2-openshift-default-cdf97ff5b-gqvbh"]
Jul 24 16:24:02.357749 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.357728    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-data\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:24:02.357799 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.357754    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-podinfo\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:24:02.357799 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.357764    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfl48\" (UniqueName: \"kubernetes.io/projected/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-kube-api-access-qfl48\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:24:02.357799 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.357773    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-envoy\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:24:02.357799 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.357782    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-workload-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:24:02.357799 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.357789    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istiod-ca-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:24:02.357799 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:02.357796    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91440df6-cce1-4af2-a2bf-b2243f0d5fc2-istio-token\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:24:03.223412 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:03.223378    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91440df6-cce1-4af2-a2bf-b2243f0d5fc2" path="/var/lib/kubelet/pods/91440df6-cce1-4af2-a2bf-b2243f0d5fc2/volumes"
Jul 24 16:24:19.094246 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:19.094165    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"]
Jul 24 16:24:19.094821 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:19.094453    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2" podUID="a3f986a0-8867-4899-8d41-fb3f6dcd3822" containerName="istio-proxy" containerID="cri-o://d4e16e6f365b21c2be863de6194c6019daa9e120e2823d483fbe9235709081b3" gracePeriod=30
Jul 24 16:24:20.029027 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.028994    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"]
Jul 24 16:24:20.029316 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.029305    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6879b4d-f68a-413b-9d2f-bd73e1b1b698" containerName="main"
Jul 24 16:24:20.029370 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.029318    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6879b4d-f68a-413b-9d2f-bd73e1b1b698" containerName="main"
Jul 24 16:24:20.029370 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.029336    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91440df6-cce1-4af2-a2bf-b2243f0d5fc2" containerName="istio-proxy"
Jul 24 16:24:20.029370 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.029342    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="91440df6-cce1-4af2-a2bf-b2243f0d5fc2" containerName="istio-proxy"
Jul 24 16:24:20.029370 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.029349    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3caafff-d976-43c5-97ce-a1dfdaf0a459" containerName="main"
Jul 24 16:24:20.029370 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.029354    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3caafff-d976-43c5-97ce-a1dfdaf0a459" containerName="main"
Jul 24 16:24:20.029370 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.029367    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91440df6-cce1-4af2-a2bf-b2243f0d5fc2" containerName="istio-proxy"
Jul 24 16:24:20.029370 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.029373    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="91440df6-cce1-4af2-a2bf-b2243f0d5fc2" containerName="istio-proxy"
Jul 24 16:24:20.029587 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.029423    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3caafff-d976-43c5-97ce-a1dfdaf0a459" containerName="main"
Jul 24 16:24:20.029587 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.029431    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="91440df6-cce1-4af2-a2bf-b2243f0d5fc2" containerName="istio-proxy"
Jul 24 16:24:20.029587 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.029437    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="91440df6-cce1-4af2-a2bf-b2243f0d5fc2" containerName="istio-proxy"
Jul 24 16:24:20.029587 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.029443    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6879b4d-f68a-413b-9d2f-bd73e1b1b698" containerName="main"
Jul 24 16:24:20.033603 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.033584    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.036148 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.036128    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-876108-openshift-default-dockercfg-7dfks\""
Jul 24 16:24:20.048980 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.048943    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"]
Jul 24 16:24:20.125031 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.124977    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istiod-ca-cert\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.125543 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.125050    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-workload-socket\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.125543 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.125081    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-token\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.125543 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.125108    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-data\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.125543 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.125132    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-podinfo\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.125543 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.125159    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfgck\" (UniqueName: \"kubernetes.io/projected/fd977a82-80c1-4198-b47e-44c65a9fcb7a-kube-api-access-bfgck\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.125543 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.125253    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-workload-certs\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.125543 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.125381    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-credential-socket\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.125543 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.125436    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-envoy\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.226739 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.226706    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-credential-socket\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.226739 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.226739    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-envoy\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.227037 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.226763    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istiod-ca-cert\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.227037 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.226790    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-workload-socket\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.227037 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.226811    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-token\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.227037 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.226969    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-data\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.227037 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.227014    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-podinfo\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.227313 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.227042    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfgck\" (UniqueName: \"kubernetes.io/projected/fd977a82-80c1-4198-b47e-44c65a9fcb7a-kube-api-access-bfgck\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.227313 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.227120    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-workload-certs\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.227313 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.227128    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-credential-socket\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.227313 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.227229    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-workload-socket\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.227313 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.227276    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-data\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.227595 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.227446    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-workload-certs\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.227662 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.227616    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istiod-ca-cert\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.229230 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.229205    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-envoy\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.229671 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.229652    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-podinfo\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.235399 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.235374    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfgck\" (UniqueName: \"kubernetes.io/projected/fd977a82-80c1-4198-b47e-44c65a9fcb7a-kube-api-access-bfgck\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.235661 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.235634    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-token\") pod \"e2e-api-a-876108-openshift-default-787f956f88-jjpkt\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") " pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.346998 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.346888    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:20.489849 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.489817    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"]
Jul 24 16:24:20.492826 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:24:20.492795    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd977a82_80c1_4198_b47e_44c65a9fcb7a.slice/crio-32377f91bb4b05908482350056ae234d7a794cf9797eb7f7c4f1dc86e4706c04 WatchSource:0}: Error finding container 32377f91bb4b05908482350056ae234d7a794cf9797eb7f7c4f1dc86e4706c04: Status 404 returned error can't find the container with id 32377f91bb4b05908482350056ae234d7a794cf9797eb7f7c4f1dc86e4706c04
Jul 24 16:24:20.494897 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.494868    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:24:20.495017 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.494967    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:24:20.495017 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:20.495002    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:24:21.397336 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:21.397297    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt" event={"ID":"fd977a82-80c1-4198-b47e-44c65a9fcb7a","Type":"ContainerStarted","Data":"141b05bae297aa2b212acb1d166b62496e88ba563b20847ad380e284281f51c5"}
Jul 24 16:24:21.397824 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:21.397364    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt" event={"ID":"fd977a82-80c1-4198-b47e-44c65a9fcb7a","Type":"ContainerStarted","Data":"32377f91bb4b05908482350056ae234d7a794cf9797eb7f7c4f1dc86e4706c04"}
Jul 24 16:24:21.418337 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:21.418266    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt" podStartSLOduration=1.418243419 podStartE2EDuration="1.418243419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:24:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:24:21.417696312 +0000 UTC m=+2340.702023935" watchObservedRunningTime="2026-07-24 16:24:21.418243419 +0000 UTC m=+2340.702571030"
Jul 24 16:24:22.347304 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:22.347266    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:22.352237 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:22.352211    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:22.402202 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:22.402170    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:22.403198 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:22.403176    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:24.349327 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.349296    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:24:24.368831 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.368730    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v47ll\" (UniqueName: \"kubernetes.io/projected/a3f986a0-8867-4899-8d41-fb3f6dcd3822-kube-api-access-v47ll\") pod \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") "
Jul 24 16:24:24.368831 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.368799    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-workload-socket\") pod \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") "
Jul 24 16:24:24.369120 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.368832    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istiod-ca-cert\") pod \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") "
Jul 24 16:24:24.369120 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.368881    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-data\") pod \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") "
Jul 24 16:24:24.369120 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.368908    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-token\") pod \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") "
Jul 24 16:24:24.369120 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.368960    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-podinfo\") pod \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") "
Jul 24 16:24:24.369120 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.369012    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-credential-socket\") pod \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") "
Jul 24 16:24:24.369120 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.369043    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-workload-certs\") pod \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") "
Jul 24 16:24:24.369120 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.369073    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-envoy\") pod \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\" (UID: \"a3f986a0-8867-4899-8d41-fb3f6dcd3822\") "
Jul 24 16:24:24.369495 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.369313    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a3f986a0-8867-4899-8d41-fb3f6dcd3822" (UID: "a3f986a0-8867-4899-8d41-fb3f6dcd3822"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:24:24.369495 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.369431    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-workload-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:24:24.369650 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.369627    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a3f986a0-8867-4899-8d41-fb3f6dcd3822" (UID: "a3f986a0-8867-4899-8d41-fb3f6dcd3822"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:24:24.370014 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.369973    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-data" (OuterVolumeSpecName: "istio-data") pod "a3f986a0-8867-4899-8d41-fb3f6dcd3822" (UID: "a3f986a0-8867-4899-8d41-fb3f6dcd3822"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:24:24.370014 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.369997    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a3f986a0-8867-4899-8d41-fb3f6dcd3822" (UID: "a3f986a0-8867-4899-8d41-fb3f6dcd3822"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:24:24.370365 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.370315    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a3f986a0-8867-4899-8d41-fb3f6dcd3822" (UID: "a3f986a0-8867-4899-8d41-fb3f6dcd3822"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:24:24.372736 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.372659    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-token" (OuterVolumeSpecName: "istio-token") pod "a3f986a0-8867-4899-8d41-fb3f6dcd3822" (UID: "a3f986a0-8867-4899-8d41-fb3f6dcd3822"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:24:24.373092 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.372817    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3f986a0-8867-4899-8d41-fb3f6dcd3822-kube-api-access-v47ll" (OuterVolumeSpecName: "kube-api-access-v47ll") pod "a3f986a0-8867-4899-8d41-fb3f6dcd3822" (UID: "a3f986a0-8867-4899-8d41-fb3f6dcd3822"). InnerVolumeSpecName "kube-api-access-v47ll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:24:24.373517 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.373486    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a3f986a0-8867-4899-8d41-fb3f6dcd3822" (UID: "a3f986a0-8867-4899-8d41-fb3f6dcd3822"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:24:24.373609 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.373485    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a3f986a0-8867-4899-8d41-fb3f6dcd3822" (UID: "a3f986a0-8867-4899-8d41-fb3f6dcd3822"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 16:24:24.414293 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.414259    2572 generic.go:356] "Generic (PLEG): container finished" podID="a3f986a0-8867-4899-8d41-fb3f6dcd3822" containerID="d4e16e6f365b21c2be863de6194c6019daa9e120e2823d483fbe9235709081b3" exitCode=0
Jul 24 16:24:24.414485 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.414316    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"
Jul 24 16:24:24.414485 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.414347    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2" event={"ID":"a3f986a0-8867-4899-8d41-fb3f6dcd3822","Type":"ContainerDied","Data":"d4e16e6f365b21c2be863de6194c6019daa9e120e2823d483fbe9235709081b3"}
Jul 24 16:24:24.414485 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.414408    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2" event={"ID":"a3f986a0-8867-4899-8d41-fb3f6dcd3822","Type":"ContainerDied","Data":"2bb4b6fb2d70656742806a23c948b982427cd6c60556d8e79fada83d6d36d09a"}
Jul 24 16:24:24.414485 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.414433    2572 scope.go:117] "RemoveContainer" containerID="d4e16e6f365b21c2be863de6194c6019daa9e120e2823d483fbe9235709081b3"
Jul 24 16:24:24.426598 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.426567    2572 scope.go:117] "RemoveContainer" containerID="e6a1db6c3f42672cb48d38dca9f90f8f73b33a2d107db90cd415d9b4fc540b92"
Jul 24 16:24:24.436499 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.436224    2572 scope.go:117] "RemoveContainer" containerID="d4e16e6f365b21c2be863de6194c6019daa9e120e2823d483fbe9235709081b3"
Jul 24 16:24:24.436696 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:24:24.436663    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4e16e6f365b21c2be863de6194c6019daa9e120e2823d483fbe9235709081b3\": container with ID starting with d4e16e6f365b21c2be863de6194c6019daa9e120e2823d483fbe9235709081b3 not found: ID does not exist" containerID="d4e16e6f365b21c2be863de6194c6019daa9e120e2823d483fbe9235709081b3"
Jul 24 16:24:24.436799 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.436711    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4e16e6f365b21c2be863de6194c6019daa9e120e2823d483fbe9235709081b3"} err="failed to get container status \"d4e16e6f365b21c2be863de6194c6019daa9e120e2823d483fbe9235709081b3\": rpc error: code = NotFound desc = could not find container \"d4e16e6f365b21c2be863de6194c6019daa9e120e2823d483fbe9235709081b3\": container with ID starting with d4e16e6f365b21c2be863de6194c6019daa9e120e2823d483fbe9235709081b3 not found: ID does not exist"
Jul 24 16:24:24.436799 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.436737    2572 scope.go:117] "RemoveContainer" containerID="e6a1db6c3f42672cb48d38dca9f90f8f73b33a2d107db90cd415d9b4fc540b92"
Jul 24 16:24:24.437213 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:24:24.437190    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6a1db6c3f42672cb48d38dca9f90f8f73b33a2d107db90cd415d9b4fc540b92\": container with ID starting with e6a1db6c3f42672cb48d38dca9f90f8f73b33a2d107db90cd415d9b4fc540b92 not found: ID does not exist" containerID="e6a1db6c3f42672cb48d38dca9f90f8f73b33a2d107db90cd415d9b4fc540b92"
Jul 24 16:24:24.437319 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.437220    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6a1db6c3f42672cb48d38dca9f90f8f73b33a2d107db90cd415d9b4fc540b92"} err="failed to get container status \"e6a1db6c3f42672cb48d38dca9f90f8f73b33a2d107db90cd415d9b4fc540b92\": rpc error: code = NotFound desc = could not find container \"e6a1db6c3f42672cb48d38dca9f90f8f73b33a2d107db90cd415d9b4fc540b92\": container with ID starting with e6a1db6c3f42672cb48d38dca9f90f8f73b33a2d107db90cd415d9b4fc540b92 not found: ID does not exist"
Jul 24 16:24:24.438598 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.438581    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"]
Jul 24 16:24:24.442538 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.442516    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-573c9b-openshift-default-98557dc94-m28s2"]
Jul 24 16:24:24.469883 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.469847    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-data\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:24:24.469883 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.469872    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-token\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:24:24.469883 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.469881    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-podinfo\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:24:24.469883 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.469895    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-credential-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:24:24.470218 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.469906    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-workload-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:24:24.470218 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.469930    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istio-envoy\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:24:24.470218 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.469942    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v47ll\" (UniqueName: \"kubernetes.io/projected/a3f986a0-8867-4899-8d41-fb3f6dcd3822-kube-api-access-v47ll\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:24:24.470218 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:24.469955    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3f986a0-8867-4899-8d41-fb3f6dcd3822-istiod-ca-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:24:25.225069 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:25.225034    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3f986a0-8867-4899-8d41-fb3f6dcd3822" path="/var/lib/kubelet/pods/a3f986a0-8867-4899-8d41-fb3f6dcd3822/volumes"
Jul 24 16:24:28.704769 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.704721    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"]
Jul 24 16:24:28.705223 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.705207    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3f986a0-8867-4899-8d41-fb3f6dcd3822" containerName="istio-proxy"
Jul 24 16:24:28.705284 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.705226    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3f986a0-8867-4899-8d41-fb3f6dcd3822" containerName="istio-proxy"
Jul 24 16:24:28.705284 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.705238    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3f986a0-8867-4899-8d41-fb3f6dcd3822" containerName="istio-proxy"
Jul 24 16:24:28.705284 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.705246    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3f986a0-8867-4899-8d41-fb3f6dcd3822" containerName="istio-proxy"
Jul 24 16:24:28.705391 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.705319    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3f986a0-8867-4899-8d41-fb3f6dcd3822" containerName="istio-proxy"
Jul 24 16:24:28.705391 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.705329    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3f986a0-8867-4899-8d41-fb3f6dcd3822" containerName="istio-proxy"
Jul 24 16:24:28.708639 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.708619    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.711269 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.711246    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-966c3e-openshift-default-dockercfg-2mjkn\""
Jul 24 16:24:28.725376 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.725340    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"]
Jul 24 16:24:28.806975 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.806899    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-workload-socket\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.806975 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.806973    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-workload-certs\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.807327 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.807040    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-podinfo\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.807327 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.807090    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-data\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.807327 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.807157    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istiod-ca-cert\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.807327 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.807185    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-token\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.807327 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.807232    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl25b\" (UniqueName: \"kubernetes.io/projected/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-kube-api-access-fl25b\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.807327 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.807294    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-credential-socket\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.807663 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.807331    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-envoy\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.908096 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.908059    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istiod-ca-cert\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.908096 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.908102    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-token\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.908350 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.908135    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl25b\" (UniqueName: \"kubernetes.io/projected/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-kube-api-access-fl25b\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.908350 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.908158    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-credential-socket\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.908350 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.908177    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-envoy\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.908506 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.908425    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-workload-socket\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.908569 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.908506    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-workload-certs\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.908569 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.908559    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-podinfo\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.908666 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.908610    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-data\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.908974 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.908947    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-workload-certs\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.908974 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.908969    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istiod-ca-cert\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.909161 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.909070    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-credential-socket\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.909377 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.909357    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-data\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.909492 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.909466    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-workload-socket\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.911198 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.911159    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-envoy\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.912444 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.912419    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-podinfo\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.922111 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.922088    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl25b\" (UniqueName: \"kubernetes.io/projected/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-kube-api-access-fl25b\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:28.925472 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:28.925447    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-token\") pod \"e2e-api-b-966c3e-openshift-default-594f876b74-vmk79\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") " pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:29.019848 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:29.019755    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:29.158338 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:29.158307    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"]
Jul 24 16:24:29.160098 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:24:29.160075    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode67e37aa_4c8f_4ada_9c92_384ce8ada32d.slice/crio-f4359663b70b4c6df4a55ae8829afcfaddd517cd2f95ff65d6ae09c5437c6a61 WatchSource:0}: Error finding container f4359663b70b4c6df4a55ae8829afcfaddd517cd2f95ff65d6ae09c5437c6a61: Status 404 returned error can't find the container with id f4359663b70b4c6df4a55ae8829afcfaddd517cd2f95ff65d6ae09c5437c6a61
Jul 24 16:24:29.162271 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:29.162243    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:24:29.162350 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:29.162305    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:24:29.162350 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:29.162337    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:24:29.440988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:29.440953    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79" event={"ID":"e67e37aa-4c8f-4ada-9c92-384ce8ada32d","Type":"ContainerStarted","Data":"1eca9b204e3d02c5504c914348af6890a3b4cb737db327c98805a71a84f6199c"}
Jul 24 16:24:29.441202 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:29.441004    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79" event={"ID":"e67e37aa-4c8f-4ada-9c92-384ce8ada32d","Type":"ContainerStarted","Data":"f4359663b70b4c6df4a55ae8829afcfaddd517cd2f95ff65d6ae09c5437c6a61"}
Jul 24 16:24:29.461155 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:29.461098    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79" podStartSLOduration=1.4610830639999999 podStartE2EDuration="1.461083064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:24:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:24:29.460150273 +0000 UTC m=+2348.744477892" watchObservedRunningTime="2026-07-24 16:24:29.461083064 +0000 UTC m=+2348.745410670"
Jul 24 16:24:30.020672 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:30.020632    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:30.026856 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:30.026822    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:30.445486 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:30.445452    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:30.446572 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:30.446554    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:38.444106 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:38.444065    2572 patch_prober.go:28] interesting pod/e2e-api-a-876108-openshift-default-787f956f88-jjpkt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 24 16:24:38.444479 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:38.444128    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt" podUID="fd977a82-80c1-4198-b47e-44c65a9fcb7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 24 16:24:40.489152 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:40.489120    2572 generic.go:356] "Generic (PLEG): container finished" podID="fd977a82-80c1-4198-b47e-44c65a9fcb7a" containerID="141b05bae297aa2b212acb1d166b62496e88ba563b20847ad380e284281f51c5" exitCode=0
Jul 24 16:24:40.489600 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:40.489199    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt" event={"ID":"fd977a82-80c1-4198-b47e-44c65a9fcb7a","Type":"ContainerDied","Data":"141b05bae297aa2b212acb1d166b62496e88ba563b20847ad380e284281f51c5"}
Jul 24 16:24:40.489600 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:40.489590    2572 scope.go:117] "RemoveContainer" containerID="141b05bae297aa2b212acb1d166b62496e88ba563b20847ad380e284281f51c5"
Jul 24 16:24:40.490574 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:40.490547    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:24:40.490638 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:40.490613    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:24:40.490682 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:40.490643    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:24:41.495023 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:41.494984    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt" event={"ID":"fd977a82-80c1-4198-b47e-44c65a9fcb7a","Type":"ContainerStarted","Data":"55a67b9ad696aebd872409b75cdca05847da24c2bba405849f2f9d1615ee3b89"}
Jul 24 16:24:42.348136 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:42.348088    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:42.348385 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:42.348359    2572 patch_prober.go:28] interesting pod/e2e-api-a-876108-openshift-default-787f956f88-jjpkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 24 16:24:42.348475 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:42.348417    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt" podUID="fd977a82-80c1-4198-b47e-44c65a9fcb7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 24 16:24:43.347809 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:43.347759    2572 patch_prober.go:28] interesting pod/e2e-api-a-876108-openshift-default-787f956f88-jjpkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 24 16:24:43.348273 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:43.347836    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt" podUID="fd977a82-80c1-4198-b47e-44c65a9fcb7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 24 16:24:44.347441 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:44.347395    2572 patch_prober.go:28] interesting pod/e2e-api-a-876108-openshift-default-787f956f88-jjpkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 24 16:24:44.347610 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:44.347479    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt" podUID="fd977a82-80c1-4198-b47e-44c65a9fcb7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 24 16:24:45.347689 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:45.347638    2572 patch_prober.go:28] interesting pod/e2e-api-a-876108-openshift-default-787f956f88-jjpkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Jul 24 16:24:45.348216 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:45.347718    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt" podUID="fd977a82-80c1-4198-b47e-44c65a9fcb7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Jul 24 16:24:46.352578 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:46.352548    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:46.353002 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:46.352822    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:46.353510 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:46.353492    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:24:46.488121 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:46.488072    2572 patch_prober.go:28] interesting pod/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 24 16:24:46.488300 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:46.488137    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79" podUID="e67e37aa-4c8f-4ada-9c92-384ce8ada32d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 24 16:24:47.531386 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:47.531304    2572 generic.go:356] "Generic (PLEG): container finished" podID="e67e37aa-4c8f-4ada-9c92-384ce8ada32d" containerID="1eca9b204e3d02c5504c914348af6890a3b4cb737db327c98805a71a84f6199c" exitCode=0
Jul 24 16:24:47.531386 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:47.531372    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79" event={"ID":"e67e37aa-4c8f-4ada-9c92-384ce8ada32d","Type":"ContainerDied","Data":"1eca9b204e3d02c5504c914348af6890a3b4cb737db327c98805a71a84f6199c"}
Jul 24 16:24:47.531899 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:47.531883    2572 scope.go:117] "RemoveContainer" containerID="1eca9b204e3d02c5504c914348af6890a3b4cb737db327c98805a71a84f6199c"
Jul 24 16:24:47.533081 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:47.533051    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:24:47.533157 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:47.533139    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:24:47.533208 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:47.533193    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:24:48.537710 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:48.537678    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79" event={"ID":"e67e37aa-4c8f-4ada-9c92-384ce8ada32d","Type":"ContainerStarted","Data":"8af48dbde02ad4ec010c5c82340ca71e60668bbf9ea837dc2447f33051a45273"}
Jul 24 16:24:49.020982 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:49.020940    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:49.021174 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:49.021155    2572 patch_prober.go:28] interesting pod/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 24 16:24:49.021224 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:49.021199    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79" podUID="e67e37aa-4c8f-4ada-9c92-384ce8ada32d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 24 16:24:50.021025 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:50.020989    2572 patch_prober.go:28] interesting pod/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 24 16:24:50.021459 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:50.021048    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79" podUID="e67e37aa-4c8f-4ada-9c92-384ce8ada32d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 24 16:24:51.020580 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:51.020534    2572 patch_prober.go:28] interesting pod/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 24 16:24:51.020770 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:51.020599    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79" podUID="e67e37aa-4c8f-4ada-9c92-384ce8ada32d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 24 16:24:52.021093 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:52.021051    2572 patch_prober.go:28] interesting pod/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Jul 24 16:24:52.021477 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:52.021124    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79" podUID="e67e37aa-4c8f-4ada-9c92-384ce8ada32d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Jul 24 16:24:53.024197 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:53.024167    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:53.024696 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:53.024548    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:24:53.025350 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:24:53.025334    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:25:11.464331 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:11.464298    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"]
Jul 24 16:25:11.464830 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:11.464539    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt" podUID="fd977a82-80c1-4198-b47e-44c65a9fcb7a" containerName="istio-proxy" containerID="cri-o://55a67b9ad696aebd872409b75cdca05847da24c2bba405849f2f9d1615ee3b89" gracePeriod=30
Jul 24 16:25:16.355347 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.355306    2572 patch_prober.go:28] interesting pod/e2e-api-a-876108-openshift-default-787f956f88-jjpkt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 16:25:16.355839 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.355386    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt" podUID="fd977a82-80c1-4198-b47e-44c65a9fcb7a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 16:25:16.673100 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.673068    2572 generic.go:356] "Generic (PLEG): container finished" podID="fd977a82-80c1-4198-b47e-44c65a9fcb7a" containerID="55a67b9ad696aebd872409b75cdca05847da24c2bba405849f2f9d1615ee3b89" exitCode=0
Jul 24 16:25:16.673300 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.673147    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt" event={"ID":"fd977a82-80c1-4198-b47e-44c65a9fcb7a","Type":"ContainerDied","Data":"55a67b9ad696aebd872409b75cdca05847da24c2bba405849f2f9d1615ee3b89"}
Jul 24 16:25:16.673300 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.673222    2572 scope.go:117] "RemoveContainer" containerID="141b05bae297aa2b212acb1d166b62496e88ba563b20847ad380e284281f51c5"
Jul 24 16:25:16.726357 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.726333    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:25:16.754762 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.754728    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-workload-socket\") pod \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") "
Jul 24 16:25:16.754960 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.754778    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-workload-certs\") pod \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") "
Jul 24 16:25:16.754960 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.754823    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfgck\" (UniqueName: \"kubernetes.io/projected/fd977a82-80c1-4198-b47e-44c65a9fcb7a-kube-api-access-bfgck\") pod \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") "
Jul 24 16:25:16.754960 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.754854    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-token\") pod \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") "
Jul 24 16:25:16.754960 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.754902    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istiod-ca-cert\") pod \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") "
Jul 24 16:25:16.754960 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.754957    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-data\") pod \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") "
Jul 24 16:25:16.755236 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.754985    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-credential-socket\") pod \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") "
Jul 24 16:25:16.755236 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.755040    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-envoy\") pod \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") "
Jul 24 16:25:16.755236 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.755065    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fd977a82-80c1-4198-b47e-44c65a9fcb7a" (UID: "fd977a82-80c1-4198-b47e-44c65a9fcb7a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:25:16.755236 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.755081    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fd977a82-80c1-4198-b47e-44c65a9fcb7a" (UID: "fd977a82-80c1-4198-b47e-44c65a9fcb7a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:25:16.755236 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.755119    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-podinfo\") pod \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\" (UID: \"fd977a82-80c1-4198-b47e-44c65a9fcb7a\") "
Jul 24 16:25:16.755510 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.755317    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fd977a82-80c1-4198-b47e-44c65a9fcb7a" (UID: "fd977a82-80c1-4198-b47e-44c65a9fcb7a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:25:16.755510 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.755331    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fd977a82-80c1-4198-b47e-44c65a9fcb7a" (UID: "fd977a82-80c1-4198-b47e-44c65a9fcb7a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:25:16.755510 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.755423    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-workload-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:25:16.755510 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.755439    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-workload-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:25:16.755510 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.755454    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istiod-ca-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:25:16.755510 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.755465    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-credential-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:25:16.756187 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.756162    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-data" (OuterVolumeSpecName: "istio-data") pod "fd977a82-80c1-4198-b47e-44c65a9fcb7a" (UID: "fd977a82-80c1-4198-b47e-44c65a9fcb7a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:25:16.757527 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.757499    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd977a82-80c1-4198-b47e-44c65a9fcb7a-kube-api-access-bfgck" (OuterVolumeSpecName: "kube-api-access-bfgck") pod "fd977a82-80c1-4198-b47e-44c65a9fcb7a" (UID: "fd977a82-80c1-4198-b47e-44c65a9fcb7a"). InnerVolumeSpecName "kube-api-access-bfgck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:25:16.757649 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.757556    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fd977a82-80c1-4198-b47e-44c65a9fcb7a" (UID: "fd977a82-80c1-4198-b47e-44c65a9fcb7a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 16:25:16.757723 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.757700    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-token" (OuterVolumeSpecName: "istio-token") pod "fd977a82-80c1-4198-b47e-44c65a9fcb7a" (UID: "fd977a82-80c1-4198-b47e-44c65a9fcb7a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:25:16.757909 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.757859    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fd977a82-80c1-4198-b47e-44c65a9fcb7a" (UID: "fd977a82-80c1-4198-b47e-44c65a9fcb7a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:25:16.856540 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.856497    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfgck\" (UniqueName: \"kubernetes.io/projected/fd977a82-80c1-4198-b47e-44c65a9fcb7a-kube-api-access-bfgck\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:25:16.856540 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.856536    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-token\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:25:16.856540 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.856547    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-data\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:25:16.856793 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.856555    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-envoy\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:25:16.856793 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:16.856563    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd977a82-80c1-4198-b47e-44c65a9fcb7a-istio-podinfo\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:25:17.679114 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:17.679082    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"
Jul 24 16:25:17.679572 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:17.679084    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt" event={"ID":"fd977a82-80c1-4198-b47e-44c65a9fcb7a","Type":"ContainerDied","Data":"32377f91bb4b05908482350056ae234d7a794cf9797eb7f7c4f1dc86e4706c04"}
Jul 24 16:25:17.679572 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:17.679220    2572 scope.go:117] "RemoveContainer" containerID="55a67b9ad696aebd872409b75cdca05847da24c2bba405849f2f9d1615ee3b89"
Jul 24 16:25:17.701642 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:17.701606    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"]
Jul 24 16:25:17.708468 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:17.708440    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-876108-openshift-default-787f956f88-jjpkt"]
Jul 24 16:25:19.223032 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:19.222999    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd977a82-80c1-4198-b47e-44c65a9fcb7a" path="/var/lib/kubelet/pods/fd977a82-80c1-4198-b47e-44c65a9fcb7a/volumes"
Jul 24 16:25:21.270697 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:21.270670    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 16:25:21.279050 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:21.279022    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 16:25:38.791373 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:38.791337    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"]
Jul 24 16:25:38.792008 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:38.791542    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79" podUID="e67e37aa-4c8f-4ada-9c92-384ce8ada32d" containerName="istio-proxy" containerID="cri-o://8af48dbde02ad4ec010c5c82340ca71e60668bbf9ea837dc2447f33051a45273" gracePeriod=30
Jul 24 16:25:39.687301 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.687265    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"]
Jul 24 16:25:39.687618 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.687604    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd977a82-80c1-4198-b47e-44c65a9fcb7a" containerName="istio-proxy"
Jul 24 16:25:39.687722 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.687622    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd977a82-80c1-4198-b47e-44c65a9fcb7a" containerName="istio-proxy"
Jul 24 16:25:39.687722 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.687654    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd977a82-80c1-4198-b47e-44c65a9fcb7a" containerName="istio-proxy"
Jul 24 16:25:39.687722 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.687660    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd977a82-80c1-4198-b47e-44c65a9fcb7a" containerName="istio-proxy"
Jul 24 16:25:39.687818 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.687730    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd977a82-80c1-4198-b47e-44c65a9fcb7a" containerName="istio-proxy"
Jul 24 16:25:39.689533 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.689516    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.692353 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.692334    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-764b09-openshift-default-dockercfg-bt7lt\""
Jul 24 16:25:39.716575 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.716541    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"]
Jul 24 16:25:39.759349 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.759305    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-data\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.759349 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.759348    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk5gj\" (UniqueName: \"kubernetes.io/projected/55e3284a-cf56-4f90-bb18-d3ee1d754149-kube-api-access-sk5gj\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.759567 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.759367    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-token\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.759567 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.759471    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-workload-certs\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.759567 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.759503    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-podinfo\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.759567 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.759554    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-credential-socket\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.759710 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.759571    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55e3284a-cf56-4f90-bb18-d3ee1d754149-istiod-ca-cert\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.759710 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.759648    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-envoy\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.759710 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.759685    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-workload-socket\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.860404 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.860358    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-workload-certs\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.860404 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.860400    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-podinfo\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.860953 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.860441    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-credential-socket\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.860953 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.860464    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55e3284a-cf56-4f90-bb18-d3ee1d754149-istiod-ca-cert\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.860953 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.860641    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-envoy\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.860953 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.860692    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-workload-socket\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.860953 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.860754    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-data\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.860953 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.860778    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk5gj\" (UniqueName: \"kubernetes.io/projected/55e3284a-cf56-4f90-bb18-d3ee1d754149-kube-api-access-sk5gj\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.860953 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.860809    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-token\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.860953 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.860885    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-workload-certs\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.861399 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.861126    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-workload-socket\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.861399 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.861136    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-data\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.861399 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.861230    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-credential-socket\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.861399 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.861381    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55e3284a-cf56-4f90-bb18-d3ee1d754149-istiod-ca-cert\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.862988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.862961    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-envoy\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.863224 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.863207    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-podinfo\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.872115 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.872093    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-token\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:39.872303 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:39.872286    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk5gj\" (UniqueName: \"kubernetes.io/projected/55e3284a-cf56-4f90-bb18-d3ee1d754149-kube-api-access-sk5gj\") pod \"e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") " pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:40.000697 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:40.000611    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:40.147055 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:40.147027    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"]
Jul 24 16:25:40.149147 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:25:40.149121    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55e3284a_cf56_4f90_bb18_d3ee1d754149.slice/crio-8503f71224ecd22df5d9602adae4790fed3ea4a1d28d9ee51df62b0d27f4773b WatchSource:0}: Error finding container 8503f71224ecd22df5d9602adae4790fed3ea4a1d28d9ee51df62b0d27f4773b: Status 404 returned error can't find the container with id 8503f71224ecd22df5d9602adae4790fed3ea4a1d28d9ee51df62b0d27f4773b
Jul 24 16:25:40.151101 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:40.151073    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:25:40.151179 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:40.151140    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:25:40.151179 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:40.151174    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:25:40.781080 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:40.781049    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b" event={"ID":"55e3284a-cf56-4f90-bb18-d3ee1d754149","Type":"ContainerStarted","Data":"e2629b6be0f608bac59a0932ec4dad2da6fa00ecb338eef8632c14150ce50f76"}
Jul 24 16:25:40.781294 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:40.781110    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b" event={"ID":"55e3284a-cf56-4f90-bb18-d3ee1d754149","Type":"ContainerStarted","Data":"8503f71224ecd22df5d9602adae4790fed3ea4a1d28d9ee51df62b0d27f4773b"}
Jul 24 16:25:40.816839 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:40.816785    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b" podStartSLOduration=1.816767038 podStartE2EDuration="1.816767038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:25:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:25:40.815745151 +0000 UTC m=+2420.100072762" watchObservedRunningTime="2026-07-24 16:25:40.816767038 +0000 UTC m=+2420.101094644"
Jul 24 16:25:41.001678 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:41.001636    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:41.006989 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:41.006961    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:41.785043 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:41.784953    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:41.785896 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:41.785877    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:44.044679 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.044655    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:25:44.208245 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.208208    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fl25b\" (UniqueName: \"kubernetes.io/projected/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-kube-api-access-fl25b\") pod \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") "
Jul 24 16:25:44.208245 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.208249    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-data\") pod \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") "
Jul 24 16:25:44.208513 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.208284    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-envoy\") pod \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") "
Jul 24 16:25:44.208513 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.208307    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-token\") pod \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") "
Jul 24 16:25:44.208513 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.208324    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-credential-socket\") pod \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") "
Jul 24 16:25:44.208513 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.208356    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-podinfo\") pod \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") "
Jul 24 16:25:44.208513 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.208381    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istiod-ca-cert\") pod \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") "
Jul 24 16:25:44.208513 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.208412    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-workload-socket\") pod \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") "
Jul 24 16:25:44.208513 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.208465    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-workload-certs\") pod \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\" (UID: \"e67e37aa-4c8f-4ada-9c92-384ce8ada32d\") "
Jul 24 16:25:44.208871 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.208817    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e67e37aa-4c8f-4ada-9c92-384ce8ada32d" (UID: "e67e37aa-4c8f-4ada-9c92-384ce8ada32d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:25:44.208975 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.208952    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e67e37aa-4c8f-4ada-9c92-384ce8ada32d" (UID: "e67e37aa-4c8f-4ada-9c92-384ce8ada32d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:25:44.209044 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.208983    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e67e37aa-4c8f-4ada-9c92-384ce8ada32d" (UID: "e67e37aa-4c8f-4ada-9c92-384ce8ada32d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:25:44.209099 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.209045    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e67e37aa-4c8f-4ada-9c92-384ce8ada32d" (UID: "e67e37aa-4c8f-4ada-9c92-384ce8ada32d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:25:44.209526 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.209494    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-data" (OuterVolumeSpecName: "istio-data") pod "e67e37aa-4c8f-4ada-9c92-384ce8ada32d" (UID: "e67e37aa-4c8f-4ada-9c92-384ce8ada32d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:25:44.211029 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.210995    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e67e37aa-4c8f-4ada-9c92-384ce8ada32d" (UID: "e67e37aa-4c8f-4ada-9c92-384ce8ada32d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:25:44.211029 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.211009    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-token" (OuterVolumeSpecName: "istio-token") pod "e67e37aa-4c8f-4ada-9c92-384ce8ada32d" (UID: "e67e37aa-4c8f-4ada-9c92-384ce8ada32d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:25:44.211288 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.211270    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e67e37aa-4c8f-4ada-9c92-384ce8ada32d" (UID: "e67e37aa-4c8f-4ada-9c92-384ce8ada32d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 16:25:44.211335 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.211302    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-kube-api-access-fl25b" (OuterVolumeSpecName: "kube-api-access-fl25b") pod "e67e37aa-4c8f-4ada-9c92-384ce8ada32d" (UID: "e67e37aa-4c8f-4ada-9c92-384ce8ada32d"). InnerVolumeSpecName "kube-api-access-fl25b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:25:44.310264 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.310215    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-podinfo\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:25:44.310264 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.310258    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istiod-ca-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:25:44.310264 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.310273    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-workload-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:25:44.310558 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.310287    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-workload-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:25:44.310558 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.310303    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fl25b\" (UniqueName: \"kubernetes.io/projected/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-kube-api-access-fl25b\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:25:44.310558 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.310329    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-data\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:25:44.310558 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.310344    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-envoy\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:25:44.310558 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.310356    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-istio-token\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:25:44.310558 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.310369    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e67e37aa-4c8f-4ada-9c92-384ce8ada32d-credential-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:25:44.800090 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.800055    2572 generic.go:356] "Generic (PLEG): container finished" podID="e67e37aa-4c8f-4ada-9c92-384ce8ada32d" containerID="8af48dbde02ad4ec010c5c82340ca71e60668bbf9ea837dc2447f33051a45273" exitCode=0
Jul 24 16:25:44.800258 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.800111    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"
Jul 24 16:25:44.800258 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.800140    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79" event={"ID":"e67e37aa-4c8f-4ada-9c92-384ce8ada32d","Type":"ContainerDied","Data":"8af48dbde02ad4ec010c5c82340ca71e60668bbf9ea837dc2447f33051a45273"}
Jul 24 16:25:44.800258 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.800199    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79" event={"ID":"e67e37aa-4c8f-4ada-9c92-384ce8ada32d","Type":"ContainerDied","Data":"f4359663b70b4c6df4a55ae8829afcfaddd517cd2f95ff65d6ae09c5437c6a61"}
Jul 24 16:25:44.800258 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.800229    2572 scope.go:117] "RemoveContainer" containerID="8af48dbde02ad4ec010c5c82340ca71e60668bbf9ea837dc2447f33051a45273"
Jul 24 16:25:44.808968 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.808945    2572 scope.go:117] "RemoveContainer" containerID="1eca9b204e3d02c5504c914348af6890a3b4cb737db327c98805a71a84f6199c"
Jul 24 16:25:44.816787 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.816770    2572 scope.go:117] "RemoveContainer" containerID="8af48dbde02ad4ec010c5c82340ca71e60668bbf9ea837dc2447f33051a45273"
Jul 24 16:25:44.817071 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:25:44.817053    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8af48dbde02ad4ec010c5c82340ca71e60668bbf9ea837dc2447f33051a45273\": container with ID starting with 8af48dbde02ad4ec010c5c82340ca71e60668bbf9ea837dc2447f33051a45273 not found: ID does not exist" containerID="8af48dbde02ad4ec010c5c82340ca71e60668bbf9ea837dc2447f33051a45273"
Jul 24 16:25:44.817123 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.817078    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8af48dbde02ad4ec010c5c82340ca71e60668bbf9ea837dc2447f33051a45273"} err="failed to get container status \"8af48dbde02ad4ec010c5c82340ca71e60668bbf9ea837dc2447f33051a45273\": rpc error: code = NotFound desc = could not find container \"8af48dbde02ad4ec010c5c82340ca71e60668bbf9ea837dc2447f33051a45273\": container with ID starting with 8af48dbde02ad4ec010c5c82340ca71e60668bbf9ea837dc2447f33051a45273 not found: ID does not exist"
Jul 24 16:25:44.817123 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.817096    2572 scope.go:117] "RemoveContainer" containerID="1eca9b204e3d02c5504c914348af6890a3b4cb737db327c98805a71a84f6199c"
Jul 24 16:25:44.817298 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:25:44.817280    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eca9b204e3d02c5504c914348af6890a3b4cb737db327c98805a71a84f6199c\": container with ID starting with 1eca9b204e3d02c5504c914348af6890a3b4cb737db327c98805a71a84f6199c not found: ID does not exist" containerID="1eca9b204e3d02c5504c914348af6890a3b4cb737db327c98805a71a84f6199c"
Jul 24 16:25:44.817339 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.817303    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eca9b204e3d02c5504c914348af6890a3b4cb737db327c98805a71a84f6199c"} err="failed to get container status \"1eca9b204e3d02c5504c914348af6890a3b4cb737db327c98805a71a84f6199c\": rpc error: code = NotFound desc = could not find container \"1eca9b204e3d02c5504c914348af6890a3b4cb737db327c98805a71a84f6199c\": container with ID starting with 1eca9b204e3d02c5504c914348af6890a3b4cb737db327c98805a71a84f6199c not found: ID does not exist"
Jul 24 16:25:44.833182 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.833157    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"]
Jul 24 16:25:44.841102 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:44.841075    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-966c3e-openshift-default-594f876b74-vmk79"]
Jul 24 16:25:45.222883 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:45.222852    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e67e37aa-4c8f-4ada-9c92-384ce8ada32d" path="/var/lib/kubelet/pods/e67e37aa-4c8f-4ada-9c92-384ce8ada32d/volumes"
Jul 24 16:25:48.097988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.097946    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"]
Jul 24 16:25:48.098461 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.098441    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e67e37aa-4c8f-4ada-9c92-384ce8ada32d" containerName="istio-proxy"
Jul 24 16:25:48.098461 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.098460    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e67e37aa-4c8f-4ada-9c92-384ce8ada32d" containerName="istio-proxy"
Jul 24 16:25:48.098581 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.098568    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e67e37aa-4c8f-4ada-9c92-384ce8ada32d" containerName="istio-proxy"
Jul 24 16:25:48.098635 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.098583    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd977a82-80c1-4198-b47e-44c65a9fcb7a" containerName="istio-proxy"
Jul 24 16:25:48.098635 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.098593    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e67e37aa-4c8f-4ada-9c92-384ce8ada32d" containerName="istio-proxy"
Jul 24 16:25:48.103206 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.103181    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.108115 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.108093    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-82adcb-openshift-default-dockercfg-58gm6\""
Jul 24 16:25:48.123355 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.123330    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"]
Jul 24 16:25:48.243485 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.243441    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istiod-ca-cert\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.243685 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.243499    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpjf6\" (UniqueName: \"kubernetes.io/projected/d9fc5ea4-4a14-41fd-addf-51b362924cd3-kube-api-access-bpjf6\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.243685 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.243616    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-data\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.243685 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.243670    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-workload-certs\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.243812 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.243705    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-workload-socket\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.243812 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.243734    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-credential-socket\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.243812 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.243750    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-podinfo\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.243812 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.243764    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-token\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.243812 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.243786    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-envoy\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.344675 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.344628    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-data\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.344675 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.344677    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-workload-certs\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.345001 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.344704    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-workload-socket\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.345001 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.344735    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-credential-socket\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.345001 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.344756    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-podinfo\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.345001 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.344777    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-token\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.345001 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.344807    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-envoy\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.345001 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.344857    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istiod-ca-cert\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.345332 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.345150    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-data\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.345332 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.345176    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-credential-socket\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.345332 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.345200    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpjf6\" (UniqueName: \"kubernetes.io/projected/d9fc5ea4-4a14-41fd-addf-51b362924cd3-kube-api-access-bpjf6\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.345332 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.345215    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-workload-certs\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.345567 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.345441    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-workload-socket\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.345625 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.345610    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istiod-ca-cert\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.347417 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.347397    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-envoy\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.347558 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.347539    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-podinfo\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.359991 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.359912    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpjf6\" (UniqueName: \"kubernetes.io/projected/d9fc5ea4-4a14-41fd-addf-51b362924cd3-kube-api-access-bpjf6\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.360306 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.360285    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-token\") pod \"e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") " pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.413988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.413673    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:48.602355 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.602086    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"]
Jul 24 16:25:48.626123 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:25:48.626094    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9fc5ea4_4a14_41fd_addf_51b362924cd3.slice/crio-c9830b026b5d7b0082ac4fd4f33bf9ae81e30290924c5c26f33d8a87ec2e6295 WatchSource:0}: Error finding container c9830b026b5d7b0082ac4fd4f33bf9ae81e30290924c5c26f33d8a87ec2e6295: Status 404 returned error can't find the container with id c9830b026b5d7b0082ac4fd4f33bf9ae81e30290924c5c26f33d8a87ec2e6295
Jul 24 16:25:48.628364 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.628335    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:25:48.628431 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.628403    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:25:48.628469 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.628433    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:25:48.826217 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.826182    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc" event={"ID":"d9fc5ea4-4a14-41fd-addf-51b362924cd3","Type":"ContainerStarted","Data":"463d7073db1d08205e1ef6fce572e1ff6c5bbad83eee6a21e2c039f9cc8a78a7"}
Jul 24 16:25:48.826411 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:48.826234    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc" event={"ID":"d9fc5ea4-4a14-41fd-addf-51b362924cd3","Type":"ContainerStarted","Data":"c9830b026b5d7b0082ac4fd4f33bf9ae81e30290924c5c26f33d8a87ec2e6295"}
Jul 24 16:25:49.415188 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:49.415148    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:49.423160 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:49.423129    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:49.466420 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:49.466366    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc" podStartSLOduration=1.466345693 podStartE2EDuration="1.466345693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:25:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:25:48.861101202 +0000 UTC m=+2428.145428808" watchObservedRunningTime="2026-07-24 16:25:49.466345693 +0000 UTC m=+2428.750673313"
Jul 24 16:25:49.830576 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:49.830480    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:49.831417 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:49.831393    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:25:55.858598 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:55.858563    2572 generic.go:356] "Generic (PLEG): container finished" podID="55e3284a-cf56-4f90-bb18-d3ee1d754149" containerID="e2629b6be0f608bac59a0932ec4dad2da6fa00ecb338eef8632c14150ce50f76" exitCode=0
Jul 24 16:25:55.859088 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:55.858637    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b" event={"ID":"55e3284a-cf56-4f90-bb18-d3ee1d754149","Type":"ContainerDied","Data":"e2629b6be0f608bac59a0932ec4dad2da6fa00ecb338eef8632c14150ce50f76"}
Jul 24 16:25:55.859088 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:55.859048    2572 scope.go:117] "RemoveContainer" containerID="e2629b6be0f608bac59a0932ec4dad2da6fa00ecb338eef8632c14150ce50f76"
Jul 24 16:25:55.860098 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:55.860070    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:25:55.860210 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:55.860140    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:25:55.860210 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:55.860175    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:25:56.785766 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:56.785718    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:56.866595 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:56.866555    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b" event={"ID":"55e3284a-cf56-4f90-bb18-d3ee1d754149","Type":"ContainerStarted","Data":"2e4ae12584ff08fbdc3511288c31afb36e597f39c42995b993a70211dd872fb7"}
Jul 24 16:25:57.002610 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:57.002574    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:25:57.003071 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:57.003046    2572 patch_prober.go:28] interesting pod/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 24 16:25:57.003274 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:57.003251    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b" podUID="55e3284a-cf56-4f90-bb18-d3ee1d754149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 24 16:25:58.002103 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:58.002066    2572 patch_prober.go:28] interesting pod/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 24 16:25:58.002699 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:58.002667    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b" podUID="55e3284a-cf56-4f90-bb18-d3ee1d754149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 24 16:25:59.001879 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:59.001840    2572 patch_prober.go:28] interesting pod/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 24 16:25:59.002187 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:25:59.002155    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b" podUID="55e3284a-cf56-4f90-bb18-d3ee1d754149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 24 16:26:00.001814 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:00.001767    2572 patch_prober.go:28] interesting pod/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body=
Jul 24 16:26:00.002163 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:00.002133    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b" podUID="55e3284a-cf56-4f90-bb18-d3ee1d754149" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused"
Jul 24 16:26:01.017320 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:01.017272    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:26:01.018516 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:01.018485    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:26:01.020566 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:01.020532    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:26:05.872196 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:05.872155    2572 patch_prober.go:28] interesting pod/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 24 16:26:05.872616 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:05.872227    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc" podUID="d9fc5ea4-4a14-41fd-addf-51b362924cd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 24 16:26:06.078769 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.078730    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"]
Jul 24 16:26:06.079157 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.079139    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e67e37aa-4c8f-4ada-9c92-384ce8ada32d" containerName="istio-proxy"
Jul 24 16:26:06.079237 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.079159    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e67e37aa-4c8f-4ada-9c92-384ce8ada32d" containerName="istio-proxy"
Jul 24 16:26:06.081317 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.081296    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.084221 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.084201    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-764b09\"/\"default-dockercfg-wl8h9\""
Jul 24 16:26:06.085378 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.085359    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-764b09\"/\"auth-test-model-764b09-kserve-self-signed-certs\""
Jul 24 16:26:06.085472 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.085386    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-764b09\"/\"openshift-service-ca.crt\""
Jul 24 16:26:06.085472 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.085400    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-764b09\"/\"kube-root-ca.crt\""
Jul 24 16:26:06.097198 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.097171    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"]
Jul 24 16:26:06.210833 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.210791    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-tmp-dir\") pod \"auth-test-model-764b09-kserve-74d4d78647-mpht2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") " pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.211071 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.210856    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq5vb\" (UniqueName: \"kubernetes.io/projected/3352c540-8271-47df-acc3-ee5836814ed2-kube-api-access-kq5vb\") pod \"auth-test-model-764b09-kserve-74d4d78647-mpht2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") " pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.211071 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.210886    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-dshm\") pod \"auth-test-model-764b09-kserve-74d4d78647-mpht2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") " pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.211071 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.210951    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-model-cache\") pod \"auth-test-model-764b09-kserve-74d4d78647-mpht2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") " pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.211071 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.210977    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3352c540-8271-47df-acc3-ee5836814ed2-tls-certs\") pod \"auth-test-model-764b09-kserve-74d4d78647-mpht2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") " pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.211255 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.211080    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-home\") pod \"auth-test-model-764b09-kserve-74d4d78647-mpht2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") " pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.312485 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.312439    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq5vb\" (UniqueName: \"kubernetes.io/projected/3352c540-8271-47df-acc3-ee5836814ed2-kube-api-access-kq5vb\") pod \"auth-test-model-764b09-kserve-74d4d78647-mpht2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") " pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.312485 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.312489    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-dshm\") pod \"auth-test-model-764b09-kserve-74d4d78647-mpht2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") " pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.312756 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.312528    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-model-cache\") pod \"auth-test-model-764b09-kserve-74d4d78647-mpht2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") " pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.312756 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.312554    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3352c540-8271-47df-acc3-ee5836814ed2-tls-certs\") pod \"auth-test-model-764b09-kserve-74d4d78647-mpht2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") " pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.312756 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.312633    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-home\") pod \"auth-test-model-764b09-kserve-74d4d78647-mpht2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") " pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.312756 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.312705    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-tmp-dir\") pod \"auth-test-model-764b09-kserve-74d4d78647-mpht2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") " pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.313107 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.313080    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-model-cache\") pod \"auth-test-model-764b09-kserve-74d4d78647-mpht2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") " pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.313379 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.313305    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-tmp-dir\") pod \"auth-test-model-764b09-kserve-74d4d78647-mpht2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") " pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.313379 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.313393    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-home\") pod \"auth-test-model-764b09-kserve-74d4d78647-mpht2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") " pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.315180 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.315162    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-dshm\") pod \"auth-test-model-764b09-kserve-74d4d78647-mpht2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") " pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.315476 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.315455    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3352c540-8271-47df-acc3-ee5836814ed2-tls-certs\") pod \"auth-test-model-764b09-kserve-74d4d78647-mpht2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") " pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.321486 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.321461    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq5vb\" (UniqueName: \"kubernetes.io/projected/3352c540-8271-47df-acc3-ee5836814ed2-kube-api-access-kq5vb\") pod \"auth-test-model-764b09-kserve-74d4d78647-mpht2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") " pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.390999 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.390946    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:06.565241 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.565202    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"]
Jul 24 16:26:06.568571 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:26:06.568541    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3352c540_8271_47df_acc3_ee5836814ed2.slice/crio-79a6442be6b8b56e1c4c10d2a962806849b37e29679c3fe272680c8fe8d9d6a1 WatchSource:0}: Error finding container 79a6442be6b8b56e1c4c10d2a962806849b37e29679c3fe272680c8fe8d9d6a1: Status 404 returned error can't find the container with id 79a6442be6b8b56e1c4c10d2a962806849b37e29679c3fe272680c8fe8d9d6a1
Jul 24 16:26:06.911967 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.911902    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2" event={"ID":"3352c540-8271-47df-acc3-ee5836814ed2","Type":"ContainerStarted","Data":"6be1be3761fab86e4a3110984730c1f784d7a5abb6701adae4257661142fdb9f"}
Jul 24 16:26:06.912422 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.911976    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2" event={"ID":"3352c540-8271-47df-acc3-ee5836814ed2","Type":"ContainerStarted","Data":"79a6442be6b8b56e1c4c10d2a962806849b37e29679c3fe272680c8fe8d9d6a1"}
Jul 24 16:26:06.936913 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:06.936861    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2" podStartSLOduration=0.936842552 podStartE2EDuration="936.842552ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:26:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:26:06.936464877 +0000 UTC m=+2446.220792483" watchObservedRunningTime="2026-07-24 16:26:06.936842552 +0000 UTC m=+2446.221170159"
Jul 24 16:26:07.917934 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:07.917876    2572 generic.go:356] "Generic (PLEG): container finished" podID="d9fc5ea4-4a14-41fd-addf-51b362924cd3" containerID="463d7073db1d08205e1ef6fce572e1ff6c5bbad83eee6a21e2c039f9cc8a78a7" exitCode=0
Jul 24 16:26:07.918360 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:07.917951    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc" event={"ID":"d9fc5ea4-4a14-41fd-addf-51b362924cd3","Type":"ContainerDied","Data":"463d7073db1d08205e1ef6fce572e1ff6c5bbad83eee6a21e2c039f9cc8a78a7"}
Jul 24 16:26:07.918403 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:07.918387    2572 scope.go:117] "RemoveContainer" containerID="463d7073db1d08205e1ef6fce572e1ff6c5bbad83eee6a21e2c039f9cc8a78a7"
Jul 24 16:26:07.919442 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:07.919413    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:26:07.919512 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:07.919496    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:26:07.919573 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:07.919558    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:26:08.925794 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:08.925749    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc" event={"ID":"d9fc5ea4-4a14-41fd-addf-51b362924cd3","Type":"ContainerStarted","Data":"e2b13224f04d9ec765fa6918c32a1654e1f20b6daf28af05e555285d13b79b3f"}
Jul 24 16:26:09.415530 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:09.415499    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:26:09.415906 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:09.415884    2572 patch_prober.go:28] interesting pod/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 24 16:26:09.416095 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:09.416072    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc" podUID="d9fc5ea4-4a14-41fd-addf-51b362924cd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 24 16:26:10.415106 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:10.415065    2572 patch_prober.go:28] interesting pod/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 24 16:26:10.415620 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:10.415134    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc" podUID="d9fc5ea4-4a14-41fd-addf-51b362924cd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 24 16:26:11.416232 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:11.416188    2572 patch_prober.go:28] interesting pod/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 24 16:26:11.416747 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:11.416268    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc" podUID="d9fc5ea4-4a14-41fd-addf-51b362924cd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 24 16:26:12.415651 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:12.415591    2572 patch_prober.go:28] interesting pod/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Jul 24 16:26:12.415874 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:12.415690    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc" podUID="d9fc5ea4-4a14-41fd-addf-51b362924cd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Jul 24 16:26:13.418448 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:13.418415    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:26:13.418830 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:13.418692    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:26:13.419390 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:13.419371    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:26:16.391245 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:16.391207    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:16.391245 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:16.391257    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:16.403825 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:16.403800    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:16.976065 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:16.976033    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:26:33.495731 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.495697    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"]
Jul 24 16:26:33.499282 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.499264    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.503876 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.503846    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-82adcb\"/\"openshift-service-ca.crt\""
Jul 24 16:26:33.504082 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.503893    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-82adcb\"/\"auth-test-model-82adcb-kserve-self-signed-certs\""
Jul 24 16:26:33.504082 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.503909    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-82adcb\"/\"default-dockercfg-5k6kb\""
Jul 24 16:26:33.504206 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.504134    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-82adcb\"/\"kube-root-ca.crt\""
Jul 24 16:26:33.509553 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.509522    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"]
Jul 24 16:26:33.657797 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.657747    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-home\") pod \"auth-test-model-82adcb-kserve-849c595899-776sh\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") " pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.658033 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.657814    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-tmp-dir\") pod \"auth-test-model-82adcb-kserve-849c595899-776sh\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") " pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.658033 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.657848    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jps4q\" (UniqueName: \"kubernetes.io/projected/17f502b9-3f54-42a3-a610-7549ff083477-kube-api-access-jps4q\") pod \"auth-test-model-82adcb-kserve-849c595899-776sh\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") " pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.658033 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.657868    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17f502b9-3f54-42a3-a610-7549ff083477-tls-certs\") pod \"auth-test-model-82adcb-kserve-849c595899-776sh\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") " pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.658033 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.657971    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-dshm\") pod \"auth-test-model-82adcb-kserve-849c595899-776sh\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") " pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.658033 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.658028    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-model-cache\") pod \"auth-test-model-82adcb-kserve-849c595899-776sh\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") " pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.759843 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.759740    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-model-cache\") pod \"auth-test-model-82adcb-kserve-849c595899-776sh\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") " pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.759843 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.759817    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-home\") pod \"auth-test-model-82adcb-kserve-849c595899-776sh\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") " pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.760121 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.759859    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-tmp-dir\") pod \"auth-test-model-82adcb-kserve-849c595899-776sh\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") " pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.760121 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.759897    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jps4q\" (UniqueName: \"kubernetes.io/projected/17f502b9-3f54-42a3-a610-7549ff083477-kube-api-access-jps4q\") pod \"auth-test-model-82adcb-kserve-849c595899-776sh\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") " pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.760121 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.759945    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17f502b9-3f54-42a3-a610-7549ff083477-tls-certs\") pod \"auth-test-model-82adcb-kserve-849c595899-776sh\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") " pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.760121 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.760010    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-dshm\") pod \"auth-test-model-82adcb-kserve-849c595899-776sh\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") " pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.760880 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.760853    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-model-cache\") pod \"auth-test-model-82adcb-kserve-849c595899-776sh\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") " pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.761034 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.760897    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-home\") pod \"auth-test-model-82adcb-kserve-849c595899-776sh\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") " pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.761034 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.760867    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-tmp-dir\") pod \"auth-test-model-82adcb-kserve-849c595899-776sh\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") " pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.762546 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.762526    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-dshm\") pod \"auth-test-model-82adcb-kserve-849c595899-776sh\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") " pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.763214 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.763196    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17f502b9-3f54-42a3-a610-7549ff083477-tls-certs\") pod \"auth-test-model-82adcb-kserve-849c595899-776sh\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") " pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.773553 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.773530    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jps4q\" (UniqueName: \"kubernetes.io/projected/17f502b9-3f54-42a3-a610-7549ff083477-kube-api-access-jps4q\") pod \"auth-test-model-82adcb-kserve-849c595899-776sh\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") " pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.809087 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.809048    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:33.960532 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:33.960505    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"]
Jul 24 16:26:33.962663 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:26:33.962636    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17f502b9_3f54_42a3_a610_7549ff083477.slice/crio-35e755e553e687eac5535258554dd333d5d2195c44438e7494ba51c69fa54155 WatchSource:0}: Error finding container 35e755e553e687eac5535258554dd333d5d2195c44438e7494ba51c69fa54155: Status 404 returned error can't find the container with id 35e755e553e687eac5535258554dd333d5d2195c44438e7494ba51c69fa54155
Jul 24 16:26:34.048601 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:34.048560    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh" event={"ID":"17f502b9-3f54-42a3-a610-7549ff083477","Type":"ContainerStarted","Data":"a970287559d0b34c8ce0719d7c2bdeeead0d002ae64bc78123398167f1704fa0"}
Jul 24 16:26:34.048778 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:34.048619    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh" event={"ID":"17f502b9-3f54-42a3-a610-7549ff083477","Type":"ContainerStarted","Data":"35e755e553e687eac5535258554dd333d5d2195c44438e7494ba51c69fa54155"}
Jul 24 16:26:34.067501 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:34.067449    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh" podStartSLOduration=1.067430038 podStartE2EDuration="1.067430038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:26:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:26:34.065371727 +0000 UTC m=+2473.349699336" watchObservedRunningTime="2026-07-24 16:26:34.067430038 +0000 UTC m=+2473.351757695"
Jul 24 16:26:43.810287 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:43.810243    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:43.810751 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:43.810302    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:43.822986 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:43.822952    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:26:44.104545 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:26:44.104452    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:28:48.666732 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:28:48.666647    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"]
Jul 24 16:28:49.776383 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:28:49.776348    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"]
Jul 24 16:29:03.687038 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:03.686973    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2" podUID="3352c540-8271-47df-acc3-ee5836814ed2" containerName="main" containerID="cri-o://6be1be3761fab86e4a3110984730c1f784d7a5abb6701adae4257661142fdb9f" gracePeriod=45
Jul 24 16:29:03.937161 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:03.937100    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:29:04.027993 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.027959    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-model-cache\") pod \"3352c540-8271-47df-acc3-ee5836814ed2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") "
Jul 24 16:29:04.028179 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.028014    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-dshm\") pod \"3352c540-8271-47df-acc3-ee5836814ed2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") "
Jul 24 16:29:04.028179 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.028045    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kq5vb\" (UniqueName: \"kubernetes.io/projected/3352c540-8271-47df-acc3-ee5836814ed2-kube-api-access-kq5vb\") pod \"3352c540-8271-47df-acc3-ee5836814ed2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") "
Jul 24 16:29:04.028179 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.028080    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3352c540-8271-47df-acc3-ee5836814ed2-tls-certs\") pod \"3352c540-8271-47df-acc3-ee5836814ed2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") "
Jul 24 16:29:04.028179 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.028097    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-tmp-dir\") pod \"3352c540-8271-47df-acc3-ee5836814ed2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") "
Jul 24 16:29:04.028179 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.028136    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-home\") pod \"3352c540-8271-47df-acc3-ee5836814ed2\" (UID: \"3352c540-8271-47df-acc3-ee5836814ed2\") "
Jul 24 16:29:04.028452 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.028289    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-model-cache" (OuterVolumeSpecName: "model-cache") pod "3352c540-8271-47df-acc3-ee5836814ed2" (UID: "3352c540-8271-47df-acc3-ee5836814ed2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:29:04.028452 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.028430    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3352c540-8271-47df-acc3-ee5836814ed2" (UID: "3352c540-8271-47df-acc3-ee5836814ed2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:29:04.028548 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.028455    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-home" (OuterVolumeSpecName: "home") pod "3352c540-8271-47df-acc3-ee5836814ed2" (UID: "3352c540-8271-47df-acc3-ee5836814ed2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:29:04.028548 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.028505    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-model-cache\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:29:04.028548 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.028529    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-tmp-dir\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:29:04.030534 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.030505    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3352c540-8271-47df-acc3-ee5836814ed2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3352c540-8271-47df-acc3-ee5836814ed2" (UID: "3352c540-8271-47df-acc3-ee5836814ed2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:29:04.030534 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.030510    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-dshm" (OuterVolumeSpecName: "dshm") pod "3352c540-8271-47df-acc3-ee5836814ed2" (UID: "3352c540-8271-47df-acc3-ee5836814ed2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:29:04.030687 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.030594    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3352c540-8271-47df-acc3-ee5836814ed2-kube-api-access-kq5vb" (OuterVolumeSpecName: "kube-api-access-kq5vb") pod "3352c540-8271-47df-acc3-ee5836814ed2" (UID: "3352c540-8271-47df-acc3-ee5836814ed2"). InnerVolumeSpecName "kube-api-access-kq5vb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:29:04.129871 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.129824    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kq5vb\" (UniqueName: \"kubernetes.io/projected/3352c540-8271-47df-acc3-ee5836814ed2-kube-api-access-kq5vb\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:29:04.129871 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.129864    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3352c540-8271-47df-acc3-ee5836814ed2-tls-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:29:04.129871 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.129875    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-home\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:29:04.129871 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.129882    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3352c540-8271-47df-acc3-ee5836814ed2-dshm\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:29:04.734319 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.734281    2572 generic.go:356] "Generic (PLEG): container finished" podID="3352c540-8271-47df-acc3-ee5836814ed2" containerID="6be1be3761fab86e4a3110984730c1f784d7a5abb6701adae4257661142fdb9f" exitCode=0
Jul 24 16:29:04.734753 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.734341    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"
Jul 24 16:29:04.734753 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.734356    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2" event={"ID":"3352c540-8271-47df-acc3-ee5836814ed2","Type":"ContainerDied","Data":"6be1be3761fab86e4a3110984730c1f784d7a5abb6701adae4257661142fdb9f"}
Jul 24 16:29:04.734753 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.734392    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2" event={"ID":"3352c540-8271-47df-acc3-ee5836814ed2","Type":"ContainerDied","Data":"79a6442be6b8b56e1c4c10d2a962806849b37e29679c3fe272680c8fe8d9d6a1"}
Jul 24 16:29:04.734753 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.734414    2572 scope.go:117] "RemoveContainer" containerID="6be1be3761fab86e4a3110984730c1f784d7a5abb6701adae4257661142fdb9f"
Jul 24 16:29:04.743726 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.743705    2572 scope.go:117] "RemoveContainer" containerID="6be1be3761fab86e4a3110984730c1f784d7a5abb6701adae4257661142fdb9f"
Jul 24 16:29:04.744004 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:29:04.743985    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6be1be3761fab86e4a3110984730c1f784d7a5abb6701adae4257661142fdb9f\": container with ID starting with 6be1be3761fab86e4a3110984730c1f784d7a5abb6701adae4257661142fdb9f not found: ID does not exist" containerID="6be1be3761fab86e4a3110984730c1f784d7a5abb6701adae4257661142fdb9f"
Jul 24 16:29:04.744059 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.744014    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6be1be3761fab86e4a3110984730c1f784d7a5abb6701adae4257661142fdb9f"} err="failed to get container status \"6be1be3761fab86e4a3110984730c1f784d7a5abb6701adae4257661142fdb9f\": rpc error: code = NotFound desc = could not find container \"6be1be3761fab86e4a3110984730c1f784d7a5abb6701adae4257661142fdb9f\": container with ID starting with 6be1be3761fab86e4a3110984730c1f784d7a5abb6701adae4257661142fdb9f not found: ID does not exist"
Jul 24 16:29:04.758548 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.758515    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"]
Jul 24 16:29:04.765145 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.765118    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-764b09/auth-test-model-764b09-kserve-74d4d78647-mpht2"]
Jul 24 16:29:04.798248 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:04.798211    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh" podUID="17f502b9-3f54-42a3-a610-7549ff083477" containerName="main" containerID="cri-o://a970287559d0b34c8ce0719d7c2bdeeead0d002ae64bc78123398167f1704fa0" gracePeriod=45
Jul 24 16:29:05.054449 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.054422    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:29:05.073478 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.073443    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"]
Jul 24 16:29:05.073967 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.073936    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3352c540-8271-47df-acc3-ee5836814ed2" containerName="main"
Jul 24 16:29:05.073967 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.073971    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3352c540-8271-47df-acc3-ee5836814ed2" containerName="main"
Jul 24 16:29:05.074070 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.073990    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17f502b9-3f54-42a3-a610-7549ff083477" containerName="main"
Jul 24 16:29:05.074070 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.073999    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="17f502b9-3f54-42a3-a610-7549ff083477" containerName="main"
Jul 24 16:29:05.074131 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.074082    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3352c540-8271-47df-acc3-ee5836814ed2" containerName="main"
Jul 24 16:29:05.074131 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.074094    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="17f502b9-3f54-42a3-a610-7549ff083477" containerName="main"
Jul 24 16:29:05.077108 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.077091    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.089493 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.089469    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-764b09\"/\"kube-root-ca.crt\""
Jul 24 16:29:05.089615 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.089595    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-764b09\"/\"sub-test-model-764b09-kserve-self-signed-certs\""
Jul 24 16:29:05.089803 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.089789    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-764b09\"/\"default-dockercfg-wl8h9\""
Jul 24 16:29:05.094953 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.094937    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-764b09\"/\"openshift-service-ca.crt\""
Jul 24 16:29:05.119498 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.119464    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"]
Jul 24 16:29:05.137267 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.137230    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-tmp-dir\") pod \"17f502b9-3f54-42a3-a610-7549ff083477\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") "
Jul 24 16:29:05.137470 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.137284    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jps4q\" (UniqueName: \"kubernetes.io/projected/17f502b9-3f54-42a3-a610-7549ff083477-kube-api-access-jps4q\") pod \"17f502b9-3f54-42a3-a610-7549ff083477\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") "
Jul 24 16:29:05.137470 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.137334    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-model-cache\") pod \"17f502b9-3f54-42a3-a610-7549ff083477\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") "
Jul 24 16:29:05.137470 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.137372    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17f502b9-3f54-42a3-a610-7549ff083477-tls-certs\") pod \"17f502b9-3f54-42a3-a610-7549ff083477\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") "
Jul 24 16:29:05.137470 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.137433    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-home\") pod \"17f502b9-3f54-42a3-a610-7549ff083477\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") "
Jul 24 16:29:05.137470 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.137456    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-dshm\") pod \"17f502b9-3f54-42a3-a610-7549ff083477\" (UID: \"17f502b9-3f54-42a3-a610-7549ff083477\") "
Jul 24 16:29:05.137765 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.137470    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "17f502b9-3f54-42a3-a610-7549ff083477" (UID: "17f502b9-3f54-42a3-a610-7549ff083477"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:29:05.137765 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.137553    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-home\") pod \"sub-test-model-764b09-kserve-86cf644fc9-nghlk\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") " pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.137765 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.137621    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-dshm\") pod \"sub-test-model-764b09-kserve-86cf644fc9-nghlk\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") " pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.137765 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.137630    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-model-cache" (OuterVolumeSpecName: "model-cache") pod "17f502b9-3f54-42a3-a610-7549ff083477" (UID: "17f502b9-3f54-42a3-a610-7549ff083477"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:29:05.137765 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.137645    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-tls-certs\") pod \"sub-test-model-764b09-kserve-86cf644fc9-nghlk\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") " pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.137765 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.137673    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ssmr\" (UniqueName: \"kubernetes.io/projected/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-kube-api-access-6ssmr\") pod \"sub-test-model-764b09-kserve-86cf644fc9-nghlk\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") " pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.137765 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.137726    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-home" (OuterVolumeSpecName: "home") pod "17f502b9-3f54-42a3-a610-7549ff083477" (UID: "17f502b9-3f54-42a3-a610-7549ff083477"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:29:05.138179 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.137825    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-tmp-dir\") pod \"sub-test-model-764b09-kserve-86cf644fc9-nghlk\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") " pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.138179 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.137964    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-model-cache\") pod \"sub-test-model-764b09-kserve-86cf644fc9-nghlk\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") " pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.138179 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.138021    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-tmp-dir\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:29:05.138179 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.138038    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-model-cache\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:29:05.138179 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.138052    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-home\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:29:05.139962 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.139911    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-dshm" (OuterVolumeSpecName: "dshm") pod "17f502b9-3f54-42a3-a610-7549ff083477" (UID: "17f502b9-3f54-42a3-a610-7549ff083477"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:29:05.140092 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.139985    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17f502b9-3f54-42a3-a610-7549ff083477-kube-api-access-jps4q" (OuterVolumeSpecName: "kube-api-access-jps4q") pod "17f502b9-3f54-42a3-a610-7549ff083477" (UID: "17f502b9-3f54-42a3-a610-7549ff083477"). InnerVolumeSpecName "kube-api-access-jps4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:29:05.140092 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.139990    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17f502b9-3f54-42a3-a610-7549ff083477-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "17f502b9-3f54-42a3-a610-7549ff083477" (UID: "17f502b9-3f54-42a3-a610-7549ff083477"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:29:05.223577 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.223532    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3352c540-8271-47df-acc3-ee5836814ed2" path="/var/lib/kubelet/pods/3352c540-8271-47df-acc3-ee5836814ed2/volumes"
Jul 24 16:29:05.238389 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.238364    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-model-cache\") pod \"sub-test-model-764b09-kserve-86cf644fc9-nghlk\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") " pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.238534 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.238397    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-home\") pod \"sub-test-model-764b09-kserve-86cf644fc9-nghlk\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") " pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.238595 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.238528    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-dshm\") pod \"sub-test-model-764b09-kserve-86cf644fc9-nghlk\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") " pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.238595 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.238557    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-tls-certs\") pod \"sub-test-model-764b09-kserve-86cf644fc9-nghlk\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") " pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.238595 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.238580    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ssmr\" (UniqueName: \"kubernetes.io/projected/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-kube-api-access-6ssmr\") pod \"sub-test-model-764b09-kserve-86cf644fc9-nghlk\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") " pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.238795 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.238632    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-tmp-dir\") pod \"sub-test-model-764b09-kserve-86cf644fc9-nghlk\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") " pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.238850 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.238809    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-home\") pod \"sub-test-model-764b09-kserve-86cf644fc9-nghlk\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") " pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.238850 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.238819    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-model-cache\") pod \"sub-test-model-764b09-kserve-86cf644fc9-nghlk\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") " pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.238970 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.238942    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jps4q\" (UniqueName: \"kubernetes.io/projected/17f502b9-3f54-42a3-a610-7549ff083477-kube-api-access-jps4q\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:29:05.239028 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.238968    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17f502b9-3f54-42a3-a610-7549ff083477-tls-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:29:05.239028 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.238984    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17f502b9-3f54-42a3-a610-7549ff083477-dshm\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:29:05.239125 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.239054    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-tmp-dir\") pod \"sub-test-model-764b09-kserve-86cf644fc9-nghlk\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") " pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.241230 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.241201    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-tls-certs\") pod \"sub-test-model-764b09-kserve-86cf644fc9-nghlk\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") " pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.241475 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.241460    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-dshm\") pod \"sub-test-model-764b09-kserve-86cf644fc9-nghlk\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") " pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.247891 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.247872    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ssmr\" (UniqueName: \"kubernetes.io/projected/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-kube-api-access-6ssmr\") pod \"sub-test-model-764b09-kserve-86cf644fc9-nghlk\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") " pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.386003 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.385864    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:05.525633 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.525608    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"]
Jul 24 16:29:05.527842 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:29:05.527811    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabbb7485_2066_492f_9d50_c0c4c0f1bdd0.slice/crio-2d650e42df713857bb64dc5981f4384b5152f7deef230b23f05129577ce02ed6 WatchSource:0}: Error finding container 2d650e42df713857bb64dc5981f4384b5152f7deef230b23f05129577ce02ed6: Status 404 returned error can't find the container with id 2d650e42df713857bb64dc5981f4384b5152f7deef230b23f05129577ce02ed6
Jul 24 16:29:05.740725 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.740694    2572 generic.go:356] "Generic (PLEG): container finished" podID="17f502b9-3f54-42a3-a610-7549ff083477" containerID="a970287559d0b34c8ce0719d7c2bdeeead0d002ae64bc78123398167f1704fa0" exitCode=0
Jul 24 16:29:05.741222 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.740737    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"
Jul 24 16:29:05.741222 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.740779    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh" event={"ID":"17f502b9-3f54-42a3-a610-7549ff083477","Type":"ContainerDied","Data":"a970287559d0b34c8ce0719d7c2bdeeead0d002ae64bc78123398167f1704fa0"}
Jul 24 16:29:05.741222 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.740815    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh" event={"ID":"17f502b9-3f54-42a3-a610-7549ff083477","Type":"ContainerDied","Data":"35e755e553e687eac5535258554dd333d5d2195c44438e7494ba51c69fa54155"}
Jul 24 16:29:05.741222 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.740834    2572 scope.go:117] "RemoveContainer" containerID="a970287559d0b34c8ce0719d7c2bdeeead0d002ae64bc78123398167f1704fa0"
Jul 24 16:29:05.742641 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.742609    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk" event={"ID":"abbb7485-2066-492f-9d50-c0c4c0f1bdd0","Type":"ContainerStarted","Data":"60501f5684531f3e14e550ebff4b2d79c5a8ac47ae1cfbf2cbfd4c249b67f6f2"}
Jul 24 16:29:05.742744 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.742654    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk" event={"ID":"abbb7485-2066-492f-9d50-c0c4c0f1bdd0","Type":"ContainerStarted","Data":"2d650e42df713857bb64dc5981f4384b5152f7deef230b23f05129577ce02ed6"}
Jul 24 16:29:05.749423 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.749407    2572 scope.go:117] "RemoveContainer" containerID="a970287559d0b34c8ce0719d7c2bdeeead0d002ae64bc78123398167f1704fa0"
Jul 24 16:29:05.749676 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:29:05.749660    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a970287559d0b34c8ce0719d7c2bdeeead0d002ae64bc78123398167f1704fa0\": container with ID starting with a970287559d0b34c8ce0719d7c2bdeeead0d002ae64bc78123398167f1704fa0 not found: ID does not exist" containerID="a970287559d0b34c8ce0719d7c2bdeeead0d002ae64bc78123398167f1704fa0"
Jul 24 16:29:05.749721 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.749683    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a970287559d0b34c8ce0719d7c2bdeeead0d002ae64bc78123398167f1704fa0"} err="failed to get container status \"a970287559d0b34c8ce0719d7c2bdeeead0d002ae64bc78123398167f1704fa0\": rpc error: code = NotFound desc = could not find container \"a970287559d0b34c8ce0719d7c2bdeeead0d002ae64bc78123398167f1704fa0\": container with ID starting with a970287559d0b34c8ce0719d7c2bdeeead0d002ae64bc78123398167f1704fa0 not found: ID does not exist"
Jul 24 16:29:05.760278 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.760252    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"]
Jul 24 16:29:05.764127 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.764104    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-82adcb/auth-test-model-82adcb-kserve-849c595899-776sh"]
Jul 24 16:29:05.779414 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:05.779371    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk" podStartSLOduration=0.779356567 podStartE2EDuration="779.356567ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:29:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:29:05.778516782 +0000 UTC m=+2625.062844390" watchObservedRunningTime="2026-07-24 16:29:05.779356567 +0000 UTC m=+2625.063684174"
Jul 24 16:29:07.222994 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:07.222959    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17f502b9-3f54-42a3-a610-7549ff083477" path="/var/lib/kubelet/pods/17f502b9-3f54-42a3-a610-7549ff083477/volumes"
Jul 24 16:29:15.386879 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:15.386838    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:15.387311 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:15.386898    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:15.399673 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:15.399642    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:15.801589 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:15.801558    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:29:38.588251 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.588217    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"]
Jul 24 16:29:38.591489 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.591467    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.594151 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.594122    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-82adcb\"/\"openshift-service-ca.crt\""
Jul 24 16:29:38.594894 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.594869    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-82adcb\"/\"default-dockercfg-5k6kb\""
Jul 24 16:29:38.595041 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.594913    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-82adcb\"/\"kube-root-ca.crt\""
Jul 24 16:29:38.595041 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.594941    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-82adcb\"/\"sub-test-model-82adcb-kserve-self-signed-certs\""
Jul 24 16:29:38.601881 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.601850    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"]
Jul 24 16:29:38.744072 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.744038    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-tmp-dir\") pod \"sub-test-model-82adcb-kserve-7fd8b5867-crd95\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") " pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.744072 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.744073    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78kp4\" (UniqueName: \"kubernetes.io/projected/f48c578a-3631-4ecc-a901-8097b5ca1bd7-kube-api-access-78kp4\") pod \"sub-test-model-82adcb-kserve-7fd8b5867-crd95\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") " pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.744344 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.744097    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-home\") pod \"sub-test-model-82adcb-kserve-7fd8b5867-crd95\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") " pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.744344 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.744188    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-model-cache\") pod \"sub-test-model-82adcb-kserve-7fd8b5867-crd95\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") " pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.744344 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.744239    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-dshm\") pod \"sub-test-model-82adcb-kserve-7fd8b5867-crd95\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") " pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.744344 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.744277    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f48c578a-3631-4ecc-a901-8097b5ca1bd7-tls-certs\") pod \"sub-test-model-82adcb-kserve-7fd8b5867-crd95\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") " pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.845387 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.845295    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-model-cache\") pod \"sub-test-model-82adcb-kserve-7fd8b5867-crd95\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") " pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.845387 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.845344    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-dshm\") pod \"sub-test-model-82adcb-kserve-7fd8b5867-crd95\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") " pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.845628 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.845396    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f48c578a-3631-4ecc-a901-8097b5ca1bd7-tls-certs\") pod \"sub-test-model-82adcb-kserve-7fd8b5867-crd95\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") " pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.845628 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.845457    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-tmp-dir\") pod \"sub-test-model-82adcb-kserve-7fd8b5867-crd95\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") " pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.845628 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.845481    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78kp4\" (UniqueName: \"kubernetes.io/projected/f48c578a-3631-4ecc-a901-8097b5ca1bd7-kube-api-access-78kp4\") pod \"sub-test-model-82adcb-kserve-7fd8b5867-crd95\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") " pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.845628 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.845512    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-home\") pod \"sub-test-model-82adcb-kserve-7fd8b5867-crd95\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") " pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.845838 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.845789    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-model-cache\") pod \"sub-test-model-82adcb-kserve-7fd8b5867-crd95\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") " pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.845838 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.845823    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-home\") pod \"sub-test-model-82adcb-kserve-7fd8b5867-crd95\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") " pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.845945 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.845890    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-tmp-dir\") pod \"sub-test-model-82adcb-kserve-7fd8b5867-crd95\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") " pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.847970 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.847903    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-dshm\") pod \"sub-test-model-82adcb-kserve-7fd8b5867-crd95\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") " pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.848117 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.848091    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f48c578a-3631-4ecc-a901-8097b5ca1bd7-tls-certs\") pod \"sub-test-model-82adcb-kserve-7fd8b5867-crd95\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") " pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.854444 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.854410    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78kp4\" (UniqueName: \"kubernetes.io/projected/f48c578a-3631-4ecc-a901-8097b5ca1bd7-kube-api-access-78kp4\") pod \"sub-test-model-82adcb-kserve-7fd8b5867-crd95\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") " pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:38.903624 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:38.903585    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:39.043179 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:39.043148    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"]
Jul 24 16:29:39.045238 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:29:39.045208    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf48c578a_3631_4ecc_a901_8097b5ca1bd7.slice/crio-721f66e28a4d00c15400d82b963a4f406809d07fcbc06a0806fbce353a2f14ac WatchSource:0}: Error finding container 721f66e28a4d00c15400d82b963a4f406809d07fcbc06a0806fbce353a2f14ac: Status 404 returned error can't find the container with id 721f66e28a4d00c15400d82b963a4f406809d07fcbc06a0806fbce353a2f14ac
Jul 24 16:29:39.902306 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:39.902254    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95" event={"ID":"f48c578a-3631-4ecc-a901-8097b5ca1bd7","Type":"ContainerStarted","Data":"e069af412144da478f6b139ded4bab54223cac0e4c120dc059181f3310df59f6"}
Jul 24 16:29:39.902717 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:39.902314    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95" event={"ID":"f48c578a-3631-4ecc-a901-8097b5ca1bd7","Type":"ContainerStarted","Data":"721f66e28a4d00c15400d82b963a4f406809d07fcbc06a0806fbce353a2f14ac"}
Jul 24 16:29:39.923715 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:39.923658    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95" podStartSLOduration=1.9236415789999999 podStartE2EDuration="1.923641579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:29:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:29:39.921374763 +0000 UTC m=+2659.205702392" watchObservedRunningTime="2026-07-24 16:29:39.923641579 +0000 UTC m=+2659.207969164"
Jul 24 16:29:48.904004 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:48.903961    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:48.904004 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:48.904010    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:48.916736 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:48.916708    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:29:48.957965 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:29:48.957937    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:30:21.297433 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:21.297343    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 16:30:21.305118 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:21.305093    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 16:30:25.393437 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:25.393397    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"]
Jul 24 16:30:26.298936 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:26.298891    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"]
Jul 24 16:30:40.413809 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.413755    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk" podUID="abbb7485-2066-492f-9d50-c0c4c0f1bdd0" containerName="main" containerID="cri-o://60501f5684531f3e14e550ebff4b2d79c5a8ac47ae1cfbf2cbfd4c249b67f6f2" gracePeriod=45
Jul 24 16:30:40.662196 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.662170    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:30:40.709169 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.709081    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-home\") pod \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") "
Jul 24 16:30:40.709169 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.709129    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-model-cache\") pod \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") "
Jul 24 16:30:40.709373 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.709182    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ssmr\" (UniqueName: \"kubernetes.io/projected/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-kube-api-access-6ssmr\") pod \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") "
Jul 24 16:30:40.709373 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.709202    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-tmp-dir\") pod \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") "
Jul 24 16:30:40.709373 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.709249    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-dshm\") pod \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") "
Jul 24 16:30:40.709373 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.709319    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-tls-certs\") pod \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\" (UID: \"abbb7485-2066-492f-9d50-c0c4c0f1bdd0\") "
Jul 24 16:30:40.709373 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.709358    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-home" (OuterVolumeSpecName: "home") pod "abbb7485-2066-492f-9d50-c0c4c0f1bdd0" (UID: "abbb7485-2066-492f-9d50-c0c4c0f1bdd0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:30:40.709642 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.709465    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-model-cache" (OuterVolumeSpecName: "model-cache") pod "abbb7485-2066-492f-9d50-c0c4c0f1bdd0" (UID: "abbb7485-2066-492f-9d50-c0c4c0f1bdd0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:30:40.709642 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.709475    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "abbb7485-2066-492f-9d50-c0c4c0f1bdd0" (UID: "abbb7485-2066-492f-9d50-c0c4c0f1bdd0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:30:40.709642 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.709601    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-tmp-dir\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:30:40.709642 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.709614    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-home\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:30:40.709642 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.709622    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-model-cache\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:30:40.711495 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.711468    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-dshm" (OuterVolumeSpecName: "dshm") pod "abbb7485-2066-492f-9d50-c0c4c0f1bdd0" (UID: "abbb7485-2066-492f-9d50-c0c4c0f1bdd0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:30:40.711495 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.711473    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-kube-api-access-6ssmr" (OuterVolumeSpecName: "kube-api-access-6ssmr") pod "abbb7485-2066-492f-9d50-c0c4c0f1bdd0" (UID: "abbb7485-2066-492f-9d50-c0c4c0f1bdd0"). InnerVolumeSpecName "kube-api-access-6ssmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:30:40.711660 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.711470    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "abbb7485-2066-492f-9d50-c0c4c0f1bdd0" (UID: "abbb7485-2066-492f-9d50-c0c4c0f1bdd0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:30:40.810890 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.810842    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-tls-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:30:40.810890 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.810881    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ssmr\" (UniqueName: \"kubernetes.io/projected/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-kube-api-access-6ssmr\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:30:40.810890 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:40.810891    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abbb7485-2066-492f-9d50-c0c4c0f1bdd0-dshm\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:30:41.176194 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.176163    2572 generic.go:356] "Generic (PLEG): container finished" podID="abbb7485-2066-492f-9d50-c0c4c0f1bdd0" containerID="60501f5684531f3e14e550ebff4b2d79c5a8ac47ae1cfbf2cbfd4c249b67f6f2" exitCode=0
Jul 24 16:30:41.176372 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.176207    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"
Jul 24 16:30:41.176372 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.176247    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk" event={"ID":"abbb7485-2066-492f-9d50-c0c4c0f1bdd0","Type":"ContainerDied","Data":"60501f5684531f3e14e550ebff4b2d79c5a8ac47ae1cfbf2cbfd4c249b67f6f2"}
Jul 24 16:30:41.176372 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.176295    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk" event={"ID":"abbb7485-2066-492f-9d50-c0c4c0f1bdd0","Type":"ContainerDied","Data":"2d650e42df713857bb64dc5981f4384b5152f7deef230b23f05129577ce02ed6"}
Jul 24 16:30:41.176372 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.176318    2572 scope.go:117] "RemoveContainer" containerID="60501f5684531f3e14e550ebff4b2d79c5a8ac47ae1cfbf2cbfd4c249b67f6f2"
Jul 24 16:30:41.185545 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.185527    2572 scope.go:117] "RemoveContainer" containerID="60501f5684531f3e14e550ebff4b2d79c5a8ac47ae1cfbf2cbfd4c249b67f6f2"
Jul 24 16:30:41.185852 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:30:41.185833    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60501f5684531f3e14e550ebff4b2d79c5a8ac47ae1cfbf2cbfd4c249b67f6f2\": container with ID starting with 60501f5684531f3e14e550ebff4b2d79c5a8ac47ae1cfbf2cbfd4c249b67f6f2 not found: ID does not exist" containerID="60501f5684531f3e14e550ebff4b2d79c5a8ac47ae1cfbf2cbfd4c249b67f6f2"
Jul 24 16:30:41.185907 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.185865    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60501f5684531f3e14e550ebff4b2d79c5a8ac47ae1cfbf2cbfd4c249b67f6f2"} err="failed to get container status \"60501f5684531f3e14e550ebff4b2d79c5a8ac47ae1cfbf2cbfd4c249b67f6f2\": rpc error: code = NotFound desc = could not find container \"60501f5684531f3e14e550ebff4b2d79c5a8ac47ae1cfbf2cbfd4c249b67f6f2\": container with ID starting with 60501f5684531f3e14e550ebff4b2d79c5a8ac47ae1cfbf2cbfd4c249b67f6f2 not found: ID does not exist"
Jul 24 16:30:41.197633 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.197554    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"]
Jul 24 16:30:41.199814 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.199791    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-764b09/sub-test-model-764b09-kserve-86cf644fc9-nghlk"]
Jul 24 16:30:41.224657 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.224626    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abbb7485-2066-492f-9d50-c0c4c0f1bdd0" path="/var/lib/kubelet/pods/abbb7485-2066-492f-9d50-c0c4c0f1bdd0/volumes"
Jul 24 16:30:41.318353 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.318285    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95" podUID="f48c578a-3631-4ecc-a901-8097b5ca1bd7" containerName="main" containerID="cri-o://e069af412144da478f6b139ded4bab54223cac0e4c120dc059181f3310df59f6" gracePeriod=45
Jul 24 16:30:41.562912 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.562885    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:30:41.618441 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.618408    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78kp4\" (UniqueName: \"kubernetes.io/projected/f48c578a-3631-4ecc-a901-8097b5ca1bd7-kube-api-access-78kp4\") pod \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") "
Jul 24 16:30:41.618638 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.618458    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-home\") pod \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") "
Jul 24 16:30:41.618638 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.618519    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-tmp-dir\") pod \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") "
Jul 24 16:30:41.618638 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.618592    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-model-cache\") pod \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") "
Jul 24 16:30:41.618638 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.618616    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-dshm\") pod \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") "
Jul 24 16:30:41.618867 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.618639    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f48c578a-3631-4ecc-a901-8097b5ca1bd7-tls-certs\") pod \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\" (UID: \"f48c578a-3631-4ecc-a901-8097b5ca1bd7\") "
Jul 24 16:30:41.618867 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.618768    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-home" (OuterVolumeSpecName: "home") pod "f48c578a-3631-4ecc-a901-8097b5ca1bd7" (UID: "f48c578a-3631-4ecc-a901-8097b5ca1bd7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:30:41.619004 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.618870    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-model-cache" (OuterVolumeSpecName: "model-cache") pod "f48c578a-3631-4ecc-a901-8097b5ca1bd7" (UID: "f48c578a-3631-4ecc-a901-8097b5ca1bd7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:30:41.619004 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.618882    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f48c578a-3631-4ecc-a901-8097b5ca1bd7" (UID: "f48c578a-3631-4ecc-a901-8097b5ca1bd7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:30:41.619079 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.619043    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-home\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:30:41.619079 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.619062    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-tmp-dir\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:30:41.619144 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.619075    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-model-cache\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:30:41.620831 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.620807    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-dshm" (OuterVolumeSpecName: "dshm") pod "f48c578a-3631-4ecc-a901-8097b5ca1bd7" (UID: "f48c578a-3631-4ecc-a901-8097b5ca1bd7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:30:41.620949 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.620890    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f48c578a-3631-4ecc-a901-8097b5ca1bd7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f48c578a-3631-4ecc-a901-8097b5ca1bd7" (UID: "f48c578a-3631-4ecc-a901-8097b5ca1bd7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:30:41.620949 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.620896    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f48c578a-3631-4ecc-a901-8097b5ca1bd7-kube-api-access-78kp4" (OuterVolumeSpecName: "kube-api-access-78kp4") pod "f48c578a-3631-4ecc-a901-8097b5ca1bd7" (UID: "f48c578a-3631-4ecc-a901-8097b5ca1bd7"). InnerVolumeSpecName "kube-api-access-78kp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:30:41.720307 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.720217    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f48c578a-3631-4ecc-a901-8097b5ca1bd7-dshm\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:30:41.720307 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.720245    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f48c578a-3631-4ecc-a901-8097b5ca1bd7-tls-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:30:41.720307 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:41.720258    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78kp4\" (UniqueName: \"kubernetes.io/projected/f48c578a-3631-4ecc-a901-8097b5ca1bd7-kube-api-access-78kp4\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:30:42.181747 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:42.181713    2572 generic.go:356] "Generic (PLEG): container finished" podID="f48c578a-3631-4ecc-a901-8097b5ca1bd7" containerID="e069af412144da478f6b139ded4bab54223cac0e4c120dc059181f3310df59f6" exitCode=0
Jul 24 16:30:42.181747 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:42.181746    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"
Jul 24 16:30:42.182079 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:42.181792    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95" event={"ID":"f48c578a-3631-4ecc-a901-8097b5ca1bd7","Type":"ContainerDied","Data":"e069af412144da478f6b139ded4bab54223cac0e4c120dc059181f3310df59f6"}
Jul 24 16:30:42.182079 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:42.181838    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95" event={"ID":"f48c578a-3631-4ecc-a901-8097b5ca1bd7","Type":"ContainerDied","Data":"721f66e28a4d00c15400d82b963a4f406809d07fcbc06a0806fbce353a2f14ac"}
Jul 24 16:30:42.182079 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:42.181863    2572 scope.go:117] "RemoveContainer" containerID="e069af412144da478f6b139ded4bab54223cac0e4c120dc059181f3310df59f6"
Jul 24 16:30:42.190632 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:42.190617    2572 scope.go:117] "RemoveContainer" containerID="e069af412144da478f6b139ded4bab54223cac0e4c120dc059181f3310df59f6"
Jul 24 16:30:42.190892 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:30:42.190874    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e069af412144da478f6b139ded4bab54223cac0e4c120dc059181f3310df59f6\": container with ID starting with e069af412144da478f6b139ded4bab54223cac0e4c120dc059181f3310df59f6 not found: ID does not exist" containerID="e069af412144da478f6b139ded4bab54223cac0e4c120dc059181f3310df59f6"
Jul 24 16:30:42.190953 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:42.190901    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e069af412144da478f6b139ded4bab54223cac0e4c120dc059181f3310df59f6"} err="failed to get container status \"e069af412144da478f6b139ded4bab54223cac0e4c120dc059181f3310df59f6\": rpc error: code = NotFound desc = could not find container \"e069af412144da478f6b139ded4bab54223cac0e4c120dc059181f3310df59f6\": container with ID starting with e069af412144da478f6b139ded4bab54223cac0e4c120dc059181f3310df59f6 not found: ID does not exist"
Jul 24 16:30:42.202464 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:42.202436    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"]
Jul 24 16:30:42.206567 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:42.206545    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-82adcb/sub-test-model-82adcb-kserve-7fd8b5867-crd95"]
Jul 24 16:30:43.223304 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:43.223272    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f48c578a-3631-4ecc-a901-8097b5ca1bd7" path="/var/lib/kubelet/pods/f48c578a-3631-4ecc-a901-8097b5ca1bd7/volumes"
Jul 24 16:30:53.600530 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.600497    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"]
Jul 24 16:30:53.600904 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.600822    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abbb7485-2066-492f-9d50-c0c4c0f1bdd0" containerName="main"
Jul 24 16:30:53.600904 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.600833    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="abbb7485-2066-492f-9d50-c0c4c0f1bdd0" containerName="main"
Jul 24 16:30:53.600904 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.600847    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f48c578a-3631-4ecc-a901-8097b5ca1bd7" containerName="main"
Jul 24 16:30:53.600904 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.600852    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f48c578a-3631-4ecc-a901-8097b5ca1bd7" containerName="main"
Jul 24 16:30:53.600904 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.600904    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f48c578a-3631-4ecc-a901-8097b5ca1bd7" containerName="main"
Jul 24 16:30:53.601115 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.600913    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="abbb7485-2066-492f-9d50-c0c4c0f1bdd0" containerName="main"
Jul 24 16:30:53.605243 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.605226    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.609169 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.609145    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-764b09\"/\"openshift-service-ca.crt\""
Jul 24 16:30:53.609326 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.609170    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-764b09\"/\"rate-test-model-764b09-kserve-self-signed-certs\""
Jul 24 16:30:53.609326 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.609210    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-764b09\"/\"default-dockercfg-wl8h9\""
Jul 24 16:30:53.609326 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.609257    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-764b09\"/\"kube-root-ca.crt\""
Jul 24 16:30:53.615029 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.615005    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"]
Jul 24 16:30:53.727888 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.727852    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-home\") pod \"rate-test-model-764b09-kserve-6856d964cf-kcxj6\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") " pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.728072 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.727906    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-tmp-dir\") pod \"rate-test-model-764b09-kserve-6856d964cf-kcxj6\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") " pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.728072 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.727978    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd0507ff-8b47-42d1-8e8c-ade7367266ae-tls-certs\") pod \"rate-test-model-764b09-kserve-6856d964cf-kcxj6\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") " pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.728072 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.727995    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-dshm\") pod \"rate-test-model-764b09-kserve-6856d964cf-kcxj6\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") " pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.728072 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.728013    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95nm5\" (UniqueName: \"kubernetes.io/projected/cd0507ff-8b47-42d1-8e8c-ade7367266ae-kube-api-access-95nm5\") pod \"rate-test-model-764b09-kserve-6856d964cf-kcxj6\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") " pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.728239 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.728112    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-model-cache\") pod \"rate-test-model-764b09-kserve-6856d964cf-kcxj6\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") " pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.829137 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.829095    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-home\") pod \"rate-test-model-764b09-kserve-6856d964cf-kcxj6\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") " pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.829323 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.829159    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-tmp-dir\") pod \"rate-test-model-764b09-kserve-6856d964cf-kcxj6\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") " pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.829323 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.829269    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd0507ff-8b47-42d1-8e8c-ade7367266ae-tls-certs\") pod \"rate-test-model-764b09-kserve-6856d964cf-kcxj6\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") " pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.829323 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.829307    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-dshm\") pod \"rate-test-model-764b09-kserve-6856d964cf-kcxj6\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") " pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.829461 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.829325    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95nm5\" (UniqueName: \"kubernetes.io/projected/cd0507ff-8b47-42d1-8e8c-ade7367266ae-kube-api-access-95nm5\") pod \"rate-test-model-764b09-kserve-6856d964cf-kcxj6\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") " pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.829461 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.829393    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-model-cache\") pod \"rate-test-model-764b09-kserve-6856d964cf-kcxj6\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") " pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.829560 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.829540    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-tmp-dir\") pod \"rate-test-model-764b09-kserve-6856d964cf-kcxj6\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") " pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.829608 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.829579    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-home\") pod \"rate-test-model-764b09-kserve-6856d964cf-kcxj6\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") " pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.829771 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.829745    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-model-cache\") pod \"rate-test-model-764b09-kserve-6856d964cf-kcxj6\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") " pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.831766 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.831738    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-dshm\") pod \"rate-test-model-764b09-kserve-6856d964cf-kcxj6\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") " pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.831961 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.831944    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd0507ff-8b47-42d1-8e8c-ade7367266ae-tls-certs\") pod \"rate-test-model-764b09-kserve-6856d964cf-kcxj6\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") " pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.838967 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.838938    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95nm5\" (UniqueName: \"kubernetes.io/projected/cd0507ff-8b47-42d1-8e8c-ade7367266ae-kube-api-access-95nm5\") pod \"rate-test-model-764b09-kserve-6856d964cf-kcxj6\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") " pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:53.915447 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:53.915414    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:30:54.056218 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:54.056178    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"]
Jul 24 16:30:54.058846 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:30:54.058820    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd0507ff_8b47_42d1_8e8c_ade7367266ae.slice/crio-5ed8a90cba0358badcc48dc16e42c324a4fd0eadf96e6d392192a24392fd6a43 WatchSource:0}: Error finding container 5ed8a90cba0358badcc48dc16e42c324a4fd0eadf96e6d392192a24392fd6a43: Status 404 returned error can't find the container with id 5ed8a90cba0358badcc48dc16e42c324a4fd0eadf96e6d392192a24392fd6a43
Jul 24 16:30:54.243955 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:54.243844    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6" event={"ID":"cd0507ff-8b47-42d1-8e8c-ade7367266ae","Type":"ContainerStarted","Data":"8cd5ef1d8d557afea858bf3736a608b0a216e1daa41047d3e19dd37956475c09"}
Jul 24 16:30:54.243955 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:54.243887    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6" event={"ID":"cd0507ff-8b47-42d1-8e8c-ade7367266ae","Type":"ContainerStarted","Data":"5ed8a90cba0358badcc48dc16e42c324a4fd0eadf96e6d392192a24392fd6a43"}
Jul 24 16:30:54.263128 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:30:54.263072    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6" podStartSLOduration=1.2630539170000001 podStartE2EDuration="1.263053917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:30:54.261672194 +0000 UTC m=+2733.545999802" watchObservedRunningTime="2026-07-24 16:30:54.263053917 +0000 UTC m=+2733.547381523"
Jul 24 16:31:03.916544 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:03.916510    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:31:03.917071 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:03.916596    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:31:03.928515 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:03.928492    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:31:04.298131 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:04.298038    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:31:10.090127 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.090095    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"]
Jul 24 16:31:10.093269 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.093253    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.096645 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.096623    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-82adcb\"/\"default-dockercfg-5k6kb\""
Jul 24 16:31:10.097635 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.097609    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-82adcb\"/\"openshift-service-ca.crt\""
Jul 24 16:31:10.097635 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.097635    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-82adcb\"/\"kube-root-ca.crt\""
Jul 24 16:31:10.097844 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.097677    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-82adcb\"/\"rate-test-model-82adcb-kserve-self-signed-certs\""
Jul 24 16:31:10.103760 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.103702    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"]
Jul 24 16:31:10.177452 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.177413    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-home\") pod \"rate-test-model-82adcb-kserve-77f9545595-qrtnh\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") " pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.177643 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.177464    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45fjq\" (UniqueName: \"kubernetes.io/projected/6601fe81-732b-49bb-92c1-1ce35d966ca6-kube-api-access-45fjq\") pod \"rate-test-model-82adcb-kserve-77f9545595-qrtnh\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") " pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.177643 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.177528    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-tmp-dir\") pod \"rate-test-model-82adcb-kserve-77f9545595-qrtnh\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") " pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.177643 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.177566    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-model-cache\") pod \"rate-test-model-82adcb-kserve-77f9545595-qrtnh\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") " pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.177643 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.177614    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-dshm\") pod \"rate-test-model-82adcb-kserve-77f9545595-qrtnh\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") " pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.177784 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.177659    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6601fe81-732b-49bb-92c1-1ce35d966ca6-tls-certs\") pod \"rate-test-model-82adcb-kserve-77f9545595-qrtnh\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") " pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.278942 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.278878    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-home\") pod \"rate-test-model-82adcb-kserve-77f9545595-qrtnh\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") " pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.279151 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.278987    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45fjq\" (UniqueName: \"kubernetes.io/projected/6601fe81-732b-49bb-92c1-1ce35d966ca6-kube-api-access-45fjq\") pod \"rate-test-model-82adcb-kserve-77f9545595-qrtnh\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") " pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.279151 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.279031    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-tmp-dir\") pod \"rate-test-model-82adcb-kserve-77f9545595-qrtnh\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") " pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.279151 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.279060    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-model-cache\") pod \"rate-test-model-82adcb-kserve-77f9545595-qrtnh\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") " pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.279151 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.279104    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-dshm\") pod \"rate-test-model-82adcb-kserve-77f9545595-qrtnh\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") " pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.279475 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.279164    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6601fe81-732b-49bb-92c1-1ce35d966ca6-tls-certs\") pod \"rate-test-model-82adcb-kserve-77f9545595-qrtnh\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") " pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.279475 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.279401    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-home\") pod \"rate-test-model-82adcb-kserve-77f9545595-qrtnh\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") " pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.279696 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.279672    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-tmp-dir\") pod \"rate-test-model-82adcb-kserve-77f9545595-qrtnh\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") " pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.280085 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.280058    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-model-cache\") pod \"rate-test-model-82adcb-kserve-77f9545595-qrtnh\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") " pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.283248 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.283227    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-dshm\") pod \"rate-test-model-82adcb-kserve-77f9545595-qrtnh\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") " pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.283386 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.283360    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6601fe81-732b-49bb-92c1-1ce35d966ca6-tls-certs\") pod \"rate-test-model-82adcb-kserve-77f9545595-qrtnh\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") " pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.291191 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.291152    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45fjq\" (UniqueName: \"kubernetes.io/projected/6601fe81-732b-49bb-92c1-1ce35d966ca6-kube-api-access-45fjq\") pod \"rate-test-model-82adcb-kserve-77f9545595-qrtnh\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") " pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.404171 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.404138    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:10.544350 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:10.544221    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"]
Jul 24 16:31:10.547149 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:31:10.547120    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6601fe81_732b_49bb_92c1_1ce35d966ca6.slice/crio-6ba2e00d814573957a2b1b79b0762831ad530887c5660229cd893c7c2dc7cd75 WatchSource:0}: Error finding container 6ba2e00d814573957a2b1b79b0762831ad530887c5660229cd893c7c2dc7cd75: Status 404 returned error can't find the container with id 6ba2e00d814573957a2b1b79b0762831ad530887c5660229cd893c7c2dc7cd75
Jul 24 16:31:11.319343 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:11.319303    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh" event={"ID":"6601fe81-732b-49bb-92c1-1ce35d966ca6","Type":"ContainerStarted","Data":"dc18a7dc309f1c8375d445b2c8e71ac20be7c655dacb82ebc15b78393e15ffa8"}
Jul 24 16:31:11.319343 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:11.319348    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh" event={"ID":"6601fe81-732b-49bb-92c1-1ce35d966ca6","Type":"ContainerStarted","Data":"6ba2e00d814573957a2b1b79b0762831ad530887c5660229cd893c7c2dc7cd75"}
Jul 24 16:31:11.338016 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:11.337960    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh" podStartSLOduration=1.337941949 podStartE2EDuration="1.337941949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:31:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:31:11.33663476 +0000 UTC m=+2750.620962367" watchObservedRunningTime="2026-07-24 16:31:11.337941949 +0000 UTC m=+2750.622269547"
Jul 24 16:31:20.404956 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:20.404900    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:20.405387 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:20.404975    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:20.417431 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:20.417399    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:21.376963 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:21.376935    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:31:56.005532 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:56.005445    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"]
Jul 24 16:31:57.079367 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:31:57.079333    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"]
Jul 24 16:32:11.025003 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.024961    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6" podUID="cd0507ff-8b47-42d1-8e8c-ade7367266ae" containerName="main" containerID="cri-o://8cd5ef1d8d557afea858bf3736a608b0a216e1daa41047d3e19dd37956475c09" gracePeriod=45
Jul 24 16:32:11.278453 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.278388    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:32:11.344676 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.344643    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd0507ff-8b47-42d1-8e8c-ade7367266ae-tls-certs\") pod \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") "
Jul 24 16:32:11.344888 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.344712    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95nm5\" (UniqueName: \"kubernetes.io/projected/cd0507ff-8b47-42d1-8e8c-ade7367266ae-kube-api-access-95nm5\") pod \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") "
Jul 24 16:32:11.344888 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.344739    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-model-cache\") pod \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") "
Jul 24 16:32:11.344888 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.344758    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-dshm\") pod \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") "
Jul 24 16:32:11.344888 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.344787    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-home\") pod \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") "
Jul 24 16:32:11.344888 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.344842    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-tmp-dir\") pod \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\" (UID: \"cd0507ff-8b47-42d1-8e8c-ade7367266ae\") "
Jul 24 16:32:11.345189 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.345098    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-model-cache" (OuterVolumeSpecName: "model-cache") pod "cd0507ff-8b47-42d1-8e8c-ade7367266ae" (UID: "cd0507ff-8b47-42d1-8e8c-ade7367266ae"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:32:11.345376 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.345274    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-home" (OuterVolumeSpecName: "home") pod "cd0507ff-8b47-42d1-8e8c-ade7367266ae" (UID: "cd0507ff-8b47-42d1-8e8c-ade7367266ae"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:32:11.345376 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.345335    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cd0507ff-8b47-42d1-8e8c-ade7367266ae" (UID: "cd0507ff-8b47-42d1-8e8c-ade7367266ae"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:32:11.345669 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.345424    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-model-cache\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:32:11.345669 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.345436    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-home\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:32:11.345669 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.345445    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-tmp-dir\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:32:11.347160 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.347130    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd0507ff-8b47-42d1-8e8c-ade7367266ae-kube-api-access-95nm5" (OuterVolumeSpecName: "kube-api-access-95nm5") pod "cd0507ff-8b47-42d1-8e8c-ade7367266ae" (UID: "cd0507ff-8b47-42d1-8e8c-ade7367266ae"). InnerVolumeSpecName "kube-api-access-95nm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:32:11.347257 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.347181    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd0507ff-8b47-42d1-8e8c-ade7367266ae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cd0507ff-8b47-42d1-8e8c-ade7367266ae" (UID: "cd0507ff-8b47-42d1-8e8c-ade7367266ae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:32:11.347257 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.347194    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-dshm" (OuterVolumeSpecName: "dshm") pod "cd0507ff-8b47-42d1-8e8c-ade7367266ae" (UID: "cd0507ff-8b47-42d1-8e8c-ade7367266ae"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:32:11.446864 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.446828    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd0507ff-8b47-42d1-8e8c-ade7367266ae-tls-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:32:11.446864 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.446856    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95nm5\" (UniqueName: \"kubernetes.io/projected/cd0507ff-8b47-42d1-8e8c-ade7367266ae-kube-api-access-95nm5\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:32:11.446864 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.446866    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd0507ff-8b47-42d1-8e8c-ade7367266ae-dshm\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:32:11.601737 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.601642    2572 generic.go:356] "Generic (PLEG): container finished" podID="cd0507ff-8b47-42d1-8e8c-ade7367266ae" containerID="8cd5ef1d8d557afea858bf3736a608b0a216e1daa41047d3e19dd37956475c09" exitCode=0
Jul 24 16:32:11.601737 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.601690    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"
Jul 24 16:32:11.601737 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.601696    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6" event={"ID":"cd0507ff-8b47-42d1-8e8c-ade7367266ae","Type":"ContainerDied","Data":"8cd5ef1d8d557afea858bf3736a608b0a216e1daa41047d3e19dd37956475c09"}
Jul 24 16:32:11.601737 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.601734    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6" event={"ID":"cd0507ff-8b47-42d1-8e8c-ade7367266ae","Type":"ContainerDied","Data":"5ed8a90cba0358badcc48dc16e42c324a4fd0eadf96e6d392192a24392fd6a43"}
Jul 24 16:32:11.602085 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.601757    2572 scope.go:117] "RemoveContainer" containerID="8cd5ef1d8d557afea858bf3736a608b0a216e1daa41047d3e19dd37956475c09"
Jul 24 16:32:11.611244 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.611224    2572 scope.go:117] "RemoveContainer" containerID="8cd5ef1d8d557afea858bf3736a608b0a216e1daa41047d3e19dd37956475c09"
Jul 24 16:32:11.611535 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:32:11.611516    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cd5ef1d8d557afea858bf3736a608b0a216e1daa41047d3e19dd37956475c09\": container with ID starting with 8cd5ef1d8d557afea858bf3736a608b0a216e1daa41047d3e19dd37956475c09 not found: ID does not exist" containerID="8cd5ef1d8d557afea858bf3736a608b0a216e1daa41047d3e19dd37956475c09"
Jul 24 16:32:11.611602 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.611548    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cd5ef1d8d557afea858bf3736a608b0a216e1daa41047d3e19dd37956475c09"} err="failed to get container status \"8cd5ef1d8d557afea858bf3736a608b0a216e1daa41047d3e19dd37956475c09\": rpc error: code = NotFound desc = could not find container \"8cd5ef1d8d557afea858bf3736a608b0a216e1daa41047d3e19dd37956475c09\": container with ID starting with 8cd5ef1d8d557afea858bf3736a608b0a216e1daa41047d3e19dd37956475c09 not found: ID does not exist"
Jul 24 16:32:11.624233 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.624204    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"]
Jul 24 16:32:11.627824 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:11.627800    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-764b09/rate-test-model-764b09-kserve-6856d964cf-kcxj6"]
Jul 24 16:32:12.098933 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.098873    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh" podUID="6601fe81-732b-49bb-92c1-1ce35d966ca6" containerName="main" containerID="cri-o://dc18a7dc309f1c8375d445b2c8e71ac20be7c655dacb82ebc15b78393e15ffa8" gracePeriod=45
Jul 24 16:32:12.360704 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.360632    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:32:12.454732 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.454690    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-model-cache\") pod \"6601fe81-732b-49bb-92c1-1ce35d966ca6\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") "
Jul 24 16:32:12.454732 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.454734    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-dshm\") pod \"6601fe81-732b-49bb-92c1-1ce35d966ca6\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") "
Jul 24 16:32:12.455038 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.454755    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-tmp-dir\") pod \"6601fe81-732b-49bb-92c1-1ce35d966ca6\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") "
Jul 24 16:32:12.455038 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.454795    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6601fe81-732b-49bb-92c1-1ce35d966ca6-tls-certs\") pod \"6601fe81-732b-49bb-92c1-1ce35d966ca6\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") "
Jul 24 16:32:12.455038 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.455016    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45fjq\" (UniqueName: \"kubernetes.io/projected/6601fe81-732b-49bb-92c1-1ce35d966ca6-kube-api-access-45fjq\") pod \"6601fe81-732b-49bb-92c1-1ce35d966ca6\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") "
Jul 24 16:32:12.455198 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.455057    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-home\") pod \"6601fe81-732b-49bb-92c1-1ce35d966ca6\" (UID: \"6601fe81-732b-49bb-92c1-1ce35d966ca6\") "
Jul 24 16:32:12.455198 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.455079    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-model-cache" (OuterVolumeSpecName: "model-cache") pod "6601fe81-732b-49bb-92c1-1ce35d966ca6" (UID: "6601fe81-732b-49bb-92c1-1ce35d966ca6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:32:12.455198 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.455102    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6601fe81-732b-49bb-92c1-1ce35d966ca6" (UID: "6601fe81-732b-49bb-92c1-1ce35d966ca6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:32:12.455358 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.455340    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-model-cache\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:32:12.455404 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.455365    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-tmp-dir\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:32:12.455452 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.455394    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-home" (OuterVolumeSpecName: "home") pod "6601fe81-732b-49bb-92c1-1ce35d966ca6" (UID: "6601fe81-732b-49bb-92c1-1ce35d966ca6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:32:12.457127 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.457108    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-dshm" (OuterVolumeSpecName: "dshm") pod "6601fe81-732b-49bb-92c1-1ce35d966ca6" (UID: "6601fe81-732b-49bb-92c1-1ce35d966ca6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:32:12.457201 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.457165    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6601fe81-732b-49bb-92c1-1ce35d966ca6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6601fe81-732b-49bb-92c1-1ce35d966ca6" (UID: "6601fe81-732b-49bb-92c1-1ce35d966ca6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:32:12.457263 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.457238    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6601fe81-732b-49bb-92c1-1ce35d966ca6-kube-api-access-45fjq" (OuterVolumeSpecName: "kube-api-access-45fjq") pod "6601fe81-732b-49bb-92c1-1ce35d966ca6" (UID: "6601fe81-732b-49bb-92c1-1ce35d966ca6"). InnerVolumeSpecName "kube-api-access-45fjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:32:12.556234 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.556190    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-dshm\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:32:12.556234 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.556226    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6601fe81-732b-49bb-92c1-1ce35d966ca6-tls-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:32:12.556234 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.556241    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45fjq\" (UniqueName: \"kubernetes.io/projected/6601fe81-732b-49bb-92c1-1ce35d966ca6-kube-api-access-45fjq\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:32:12.556485 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.556253    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6601fe81-732b-49bb-92c1-1ce35d966ca6-home\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:32:12.607131 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.607093    2572 generic.go:356] "Generic (PLEG): container finished" podID="6601fe81-732b-49bb-92c1-1ce35d966ca6" containerID="dc18a7dc309f1c8375d445b2c8e71ac20be7c655dacb82ebc15b78393e15ffa8" exitCode=0
Jul 24 16:32:12.607131 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.607135    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"
Jul 24 16:32:12.607397 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.607179    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh" event={"ID":"6601fe81-732b-49bb-92c1-1ce35d966ca6","Type":"ContainerDied","Data":"dc18a7dc309f1c8375d445b2c8e71ac20be7c655dacb82ebc15b78393e15ffa8"}
Jul 24 16:32:12.607397 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.607230    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh" event={"ID":"6601fe81-732b-49bb-92c1-1ce35d966ca6","Type":"ContainerDied","Data":"6ba2e00d814573957a2b1b79b0762831ad530887c5660229cd893c7c2dc7cd75"}
Jul 24 16:32:12.607397 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.607253    2572 scope.go:117] "RemoveContainer" containerID="dc18a7dc309f1c8375d445b2c8e71ac20be7c655dacb82ebc15b78393e15ffa8"
Jul 24 16:32:12.616399 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.616381    2572 scope.go:117] "RemoveContainer" containerID="dc18a7dc309f1c8375d445b2c8e71ac20be7c655dacb82ebc15b78393e15ffa8"
Jul 24 16:32:12.616694 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:32:12.616675    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc18a7dc309f1c8375d445b2c8e71ac20be7c655dacb82ebc15b78393e15ffa8\": container with ID starting with dc18a7dc309f1c8375d445b2c8e71ac20be7c655dacb82ebc15b78393e15ffa8 not found: ID does not exist" containerID="dc18a7dc309f1c8375d445b2c8e71ac20be7c655dacb82ebc15b78393e15ffa8"
Jul 24 16:32:12.616760 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.616705    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc18a7dc309f1c8375d445b2c8e71ac20be7c655dacb82ebc15b78393e15ffa8"} err="failed to get container status \"dc18a7dc309f1c8375d445b2c8e71ac20be7c655dacb82ebc15b78393e15ffa8\": rpc error: code = NotFound desc = could not find container \"dc18a7dc309f1c8375d445b2c8e71ac20be7c655dacb82ebc15b78393e15ffa8\": container with ID starting with dc18a7dc309f1c8375d445b2c8e71ac20be7c655dacb82ebc15b78393e15ffa8 not found: ID does not exist"
Jul 24 16:32:12.628595 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.628560    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"]
Jul 24 16:32:12.632877 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:12.632848    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-82adcb/rate-test-model-82adcb-kserve-77f9545595-qrtnh"]
Jul 24 16:32:13.223759 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:13.223720    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6601fe81-732b-49bb-92c1-1ce35d966ca6" path="/var/lib/kubelet/pods/6601fe81-732b-49bb-92c1-1ce35d966ca6/volumes"
Jul 24 16:32:13.224232 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:13.224216    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd0507ff-8b47-42d1-8e8c-ade7367266ae" path="/var/lib/kubelet/pods/cd0507ff-8b47-42d1-8e8c-ade7367266ae/volumes"
Jul 24 16:32:13.383999 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:13.383965    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7462-139787518073280"]
Jul 24 16:32:13.384307 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:13.384295    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd0507ff-8b47-42d1-8e8c-ade7367266ae" containerName="main"
Jul 24 16:32:13.384351 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:13.384309    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd0507ff-8b47-42d1-8e8c-ade7367266ae" containerName="main"
Jul 24 16:32:13.384351 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:13.384322    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6601fe81-732b-49bb-92c1-1ce35d966ca6" containerName="main"
Jul 24 16:32:13.384351 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:13.384327    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6601fe81-732b-49bb-92c1-1ce35d966ca6" containerName="main"
Jul 24 16:32:13.384449 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:13.384382    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6601fe81-732b-49bb-92c1-1ce35d966ca6" containerName="main"
Jul 24 16:32:13.384449 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:13.384388    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd0507ff-8b47-42d1-8e8c-ade7367266ae" containerName="main"
Jul 24 16:32:13.388571 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:13.388555    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-139787518073280"
Jul 24 16:32:13.391069 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:13.391041    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7zspr\""
Jul 24 16:32:13.394713 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:13.394684    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7462-139787518073280"]
Jul 24 16:32:13.464311 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:13.464274    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf4vn\" (UniqueName: \"kubernetes.io/projected/b85e28cb-f35b-4533-bf11-1b705ecba065-kube-api-access-gf4vn\") pod \"test-curl-7462-139787518073280\" (UID: \"b85e28cb-f35b-4533-bf11-1b705ecba065\") " pod="opendatahub/test-curl-7462-139787518073280"
Jul 24 16:32:13.564956 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:13.564844    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf4vn\" (UniqueName: \"kubernetes.io/projected/b85e28cb-f35b-4533-bf11-1b705ecba065-kube-api-access-gf4vn\") pod \"test-curl-7462-139787518073280\" (UID: \"b85e28cb-f35b-4533-bf11-1b705ecba065\") " pod="opendatahub/test-curl-7462-139787518073280"
Jul 24 16:32:13.575291 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:13.575258    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf4vn\" (UniqueName: \"kubernetes.io/projected/b85e28cb-f35b-4533-bf11-1b705ecba065-kube-api-access-gf4vn\") pod \"test-curl-7462-139787518073280\" (UID: \"b85e28cb-f35b-4533-bf11-1b705ecba065\") " pod="opendatahub/test-curl-7462-139787518073280"
Jul 24 16:32:13.698623 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:13.698586    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-139787518073280"
Jul 24 16:32:13.833067 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:13.833042    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7462-139787518073280"]
Jul 24 16:32:13.835304 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:32:13.835275    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb85e28cb_f35b_4533_bf11_1b705ecba065.slice/crio-8c02a12f5a00cfa688bb7bea2a69cfa0297f50b96a20bb3c20b20a86fdabaf2c WatchSource:0}: Error finding container 8c02a12f5a00cfa688bb7bea2a69cfa0297f50b96a20bb3c20b20a86fdabaf2c: Status 404 returned error can't find the container with id 8c02a12f5a00cfa688bb7bea2a69cfa0297f50b96a20bb3c20b20a86fdabaf2c
Jul 24 16:32:13.836624 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:13.836602    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 24 16:32:14.620450 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:14.620415    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-139787518073280" event={"ID":"b85e28cb-f35b-4533-bf11-1b705ecba065","Type":"ContainerStarted","Data":"4548fa32ab949ffaa3ae544fb2c9f55605680d1155a0be3283097f8d59d3373b"}
Jul 24 16:32:14.620858 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:14.620461    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-139787518073280" event={"ID":"b85e28cb-f35b-4533-bf11-1b705ecba065","Type":"ContainerStarted","Data":"8c02a12f5a00cfa688bb7bea2a69cfa0297f50b96a20bb3c20b20a86fdabaf2c"}
Jul 24 16:32:14.635619 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:14.635571    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7462-139787518073280" podStartSLOduration=0.951089403 podStartE2EDuration="1.635549134s" totalImagesPullingTime="684.459731ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:32:14.634651025 +0000 UTC m=+2813.918978658" watchObservedRunningTime="2026-07-24 16:32:14.635549134 +0000 UTC m=+2813.919876741"
Jul 24 16:32:14.761194 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:14.761162    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7462-139787518073280_b85e28cb-f35b-4533-bf11-1b705ecba065/test-curl-7462-139787518073280/0.log"
Jul 24 16:32:15.626360 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:15.626330    2572 generic.go:356] "Generic (PLEG): container finished" podID="b85e28cb-f35b-4533-bf11-1b705ecba065" containerID="4548fa32ab949ffaa3ae544fb2c9f55605680d1155a0be3283097f8d59d3373b" exitCode=0
Jul 24 16:32:15.626765 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:15.626384    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7462-139787518073280" event={"ID":"b85e28cb-f35b-4533-bf11-1b705ecba065","Type":"ContainerDied","Data":"4548fa32ab949ffaa3ae544fb2c9f55605680d1155a0be3283097f8d59d3373b"}
Jul 24 16:32:16.767461 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:16.767434    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-139787518073280"
Jul 24 16:32:16.799750 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:16.799710    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7462-139787518073280"]
Jul 24 16:32:16.805393 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:16.805350    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7462-139787518073280"]
Jul 24 16:32:16.898196 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:16.898162    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gf4vn\" (UniqueName: \"kubernetes.io/projected/b85e28cb-f35b-4533-bf11-1b705ecba065-kube-api-access-gf4vn\") pod \"b85e28cb-f35b-4533-bf11-1b705ecba065\" (UID: \"b85e28cb-f35b-4533-bf11-1b705ecba065\") "
Jul 24 16:32:16.900584 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:16.900546    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b85e28cb-f35b-4533-bf11-1b705ecba065-kube-api-access-gf4vn" (OuterVolumeSpecName: "kube-api-access-gf4vn") pod "b85e28cb-f35b-4533-bf11-1b705ecba065" (UID: "b85e28cb-f35b-4533-bf11-1b705ecba065"). InnerVolumeSpecName "kube-api-access-gf4vn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:32:16.999373 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:16.999331    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gf4vn\" (UniqueName: \"kubernetes.io/projected/b85e28cb-f35b-4533-bf11-1b705ecba065-kube-api-access-gf4vn\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:32:17.222895 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.222814    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b85e28cb-f35b-4533-bf11-1b705ecba065" path="/var/lib/kubelet/pods/b85e28cb-f35b-4533-bf11-1b705ecba065/volumes"
Jul 24 16:32:17.637004 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.636890    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7462-139787518073280"
Jul 24 16:32:17.637004 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.636959    2572 scope.go:117] "RemoveContainer" containerID="4548fa32ab949ffaa3ae544fb2c9f55605680d1155a0be3283097f8d59d3373b"
Jul 24 16:32:17.797605 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.797561    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"]
Jul 24 16:32:17.798148 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.798127    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b85e28cb-f35b-4533-bf11-1b705ecba065" containerName="test-curl-7462-139787518073280"
Jul 24 16:32:17.798231 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.798150    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b85e28cb-f35b-4533-bf11-1b705ecba065" containerName="test-curl-7462-139787518073280"
Jul 24 16:32:17.798297 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.798240    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b85e28cb-f35b-4533-bf11-1b705ecba065" containerName="test-curl-7462-139787518073280"
Jul 24 16:32:17.802637 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.802614    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:17.806052 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.806030    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-0107ee-openshift-default-dockercfg-8jt5p\""
Jul 24 16:32:17.833055 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.833021    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"]
Jul 24 16:32:17.908235 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.908197    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-workload-socket\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:17.908410 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.908252    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-workload-certs\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:17.908410 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.908271    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdf7137c-282c-4c64-84e7-277e09187336-istio-token\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:17.908410 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.908290    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdf7137c-282c-4c64-84e7-277e09187336-istiod-ca-cert\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:17.908410 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.908334    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-credential-socket\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:17.908410 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.908352    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-istio-envoy\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:17.908644 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.908421    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdf7137c-282c-4c64-84e7-277e09187336-istio-podinfo\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:17.908644 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.908449    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plpqh\" (UniqueName: \"kubernetes.io/projected/cdf7137c-282c-4c64-84e7-277e09187336-kube-api-access-plpqh\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:17.908644 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:17.908479    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-istio-data\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.009617 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.009574    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-workload-socket\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.009968 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.009904    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-workload-certs\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.010105 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.009977    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdf7137c-282c-4c64-84e7-277e09187336-istio-token\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.010105 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.010004    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdf7137c-282c-4c64-84e7-277e09187336-istiod-ca-cert\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.010105 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.010053    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-credential-socket\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.010105 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.010078    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-istio-envoy\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.010344 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.010111    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-workload-socket\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.010344 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.010122    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdf7137c-282c-4c64-84e7-277e09187336-istio-podinfo\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.010344 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.010150    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plpqh\" (UniqueName: \"kubernetes.io/projected/cdf7137c-282c-4c64-84e7-277e09187336-kube-api-access-plpqh\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.010344 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.010290    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-workload-certs\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.010344 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.010310    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-istio-data\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.010626 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.010400    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-credential-socket\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.010729 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.010694    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-istio-data\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.011034 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.010983    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdf7137c-282c-4c64-84e7-277e09187336-istiod-ca-cert\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.013803 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.013727    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-istio-envoy\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.014537 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.014512    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdf7137c-282c-4c64-84e7-277e09187336-istio-podinfo\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.020274 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.020243    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plpqh\" (UniqueName: \"kubernetes.io/projected/cdf7137c-282c-4c64-84e7-277e09187336-kube-api-access-plpqh\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.020390 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.020335    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdf7137c-282c-4c64-84e7-277e09187336-istio-token\") pod \"e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") " pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.113238 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.113190    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:18.261410 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.261382    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"]
Jul 24 16:32:18.263459 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:32:18.263429    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdf7137c_282c_4c64_84e7_277e09187336.slice/crio-9056f5ed44ccc0dc5210af103876da49d58a4a0d0c2cd9ac68ab8743557baeed WatchSource:0}: Error finding container 9056f5ed44ccc0dc5210af103876da49d58a4a0d0c2cd9ac68ab8743557baeed: Status 404 returned error can't find the container with id 9056f5ed44ccc0dc5210af103876da49d58a4a0d0c2cd9ac68ab8743557baeed
Jul 24 16:32:18.265771 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.265729    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:32:18.265909 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.265804    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:32:18.265909 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.265838    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:32:18.647299 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.647265    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n" event={"ID":"cdf7137c-282c-4c64-84e7-277e09187336","Type":"ContainerStarted","Data":"f06949d163691aa7893e4895fd5e8424ebadc936844b1706fb4bc9841c19cdeb"}
Jul 24 16:32:18.647507 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.647316    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n" event={"ID":"cdf7137c-282c-4c64-84e7-277e09187336","Type":"ContainerStarted","Data":"9056f5ed44ccc0dc5210af103876da49d58a4a0d0c2cd9ac68ab8743557baeed"}
Jul 24 16:32:18.672290 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:18.672229    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n" podStartSLOduration=1.6722110639999999 podStartE2EDuration="1.672211064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:32:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:32:18.669577497 +0000 UTC m=+2817.953905125" watchObservedRunningTime="2026-07-24 16:32:18.672211064 +0000 UTC m=+2817.956538670"
Jul 24 16:32:19.114166 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:19.114065    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:19.119310 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:19.119281    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:19.652361 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:19.652331    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:19.653227 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:19.653211    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:36.468819 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.468765    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"]
Jul 24 16:32:36.473061 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.473032    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.475869 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.475843    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-52d140-openshift-default-dockercfg-84jfj\""
Jul 24 16:32:36.487494 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.487465    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"]
Jul 24 16:32:36.582004 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.581623    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zzlk\" (UniqueName: \"kubernetes.io/projected/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-kube-api-access-5zzlk\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.582343 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.582309    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-workload-socket\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.582481 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.582376    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-data\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.582481 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.582401    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-podinfo\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.582481 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.582427    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-token\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.582481 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.582465    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-workload-certs\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.582710 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.582491    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istiod-ca-cert\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.582710 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.582556    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-credential-socket\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.582710 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.582587    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-envoy\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.684014 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.683980    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zzlk\" (UniqueName: \"kubernetes.io/projected/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-kube-api-access-5zzlk\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.684014 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.684021    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-workload-socket\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.684240 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.684050    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-data\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.684240 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.684066    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-podinfo\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.684240 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.684219    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-token\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.684388 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.684344    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-workload-certs\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.684388 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.684376    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istiod-ca-cert\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.684494 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.684441    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-credential-socket\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.684494 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.684472    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-envoy\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.684597 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.684512    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-data\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.684597 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.684549    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-workload-socket\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.684729 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.684704    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-workload-certs\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.685061 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.684838    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-credential-socket\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.685264 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.685239    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istiod-ca-cert\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.686879 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.686861    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-podinfo\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.686998 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.686984    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-envoy\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.696094 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.696034    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zzlk\" (UniqueName: \"kubernetes.io/projected/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-kube-api-access-5zzlk\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.696537 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.696494    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-token\") pod \"e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") " pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.786414 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.786321    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:36.932234 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.932205    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"]
Jul 24 16:32:36.933949 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:32:36.933887    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb5ab5d_6d19_436a_8119_f5d2bce30bb4.slice/crio-5416005293f410abb1bb594d35008ed9e27018ac456c76cd926b31def5955996 WatchSource:0}: Error finding container 5416005293f410abb1bb594d35008ed9e27018ac456c76cd926b31def5955996: Status 404 returned error can't find the container with id 5416005293f410abb1bb594d35008ed9e27018ac456c76cd926b31def5955996
Jul 24 16:32:36.936158 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.936123    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:32:36.936248 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.936213    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:32:36.936288 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:36.936257    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:32:37.748455 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:37.748413    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2" event={"ID":"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4","Type":"ContainerStarted","Data":"844bb2d2fdc3ee3336fda28387cd63c325c8f4403aaeecaf6264e312bc021f6c"}
Jul 24 16:32:37.749012 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:37.748489    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2" event={"ID":"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4","Type":"ContainerStarted","Data":"5416005293f410abb1bb594d35008ed9e27018ac456c76cd926b31def5955996"}
Jul 24 16:32:37.770359 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:37.770308    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2" podStartSLOduration=1.770290419 podStartE2EDuration="1.770290419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:32:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:32:37.768332865 +0000 UTC m=+2837.052660500" watchObservedRunningTime="2026-07-24 16:32:37.770290419 +0000 UTC m=+2837.054618025"
Jul 24 16:32:37.787367 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:37.787322    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:37.792194 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:37.792163    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:38.754702 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:38.754670    2572 generic.go:356] "Generic (PLEG): container finished" podID="cdf7137c-282c-4c64-84e7-277e09187336" containerID="f06949d163691aa7893e4895fd5e8424ebadc936844b1706fb4bc9841c19cdeb" exitCode=0
Jul 24 16:32:38.755261 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:38.754753    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n" event={"ID":"cdf7137c-282c-4c64-84e7-277e09187336","Type":"ContainerDied","Data":"f06949d163691aa7893e4895fd5e8424ebadc936844b1706fb4bc9841c19cdeb"}
Jul 24 16:32:38.755261 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:38.754913    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:38.755261 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:38.755243    2572 scope.go:117] "RemoveContainer" containerID="f06949d163691aa7893e4895fd5e8424ebadc936844b1706fb4bc9841c19cdeb"
Jul 24 16:32:38.756150 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:38.756131    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:38.756348 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:38.756322    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:32:38.756439 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:38.756388    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:32:38.756439 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:38.756417    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:32:39.760603 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:39.760561    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n" event={"ID":"cdf7137c-282c-4c64-84e7-277e09187336","Type":"ContainerStarted","Data":"fc05666100a5204ff2e2dc0aadc65c9793ba8afd075cddcf2f8cb275c6a3f516"}
Jul 24 16:32:40.114094 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:40.113984    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:40.114422 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:40.114392    2572 patch_prober.go:28] interesting pod/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 24 16:32:40.114497 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:40.114453    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n" podUID="cdf7137c-282c-4c64-84e7-277e09187336" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 24 16:32:41.114112 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:41.114066    2572 patch_prober.go:28] interesting pod/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 24 16:32:41.114666 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:41.114147    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n" podUID="cdf7137c-282c-4c64-84e7-277e09187336" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 24 16:32:42.113840 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:42.113788    2572 patch_prober.go:28] interesting pod/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 24 16:32:42.114098 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:42.113863    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n" podUID="cdf7137c-282c-4c64-84e7-277e09187336" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 24 16:32:43.114896 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:43.114415    2572 patch_prober.go:28] interesting pod/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 24 16:32:43.114896 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:43.114497    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n" podUID="cdf7137c-282c-4c64-84e7-277e09187336" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 24 16:32:44.114269 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:44.114227    2572 patch_prober.go:28] interesting pod/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 24 16:32:44.114546 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:44.114519    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n" podUID="cdf7137c-282c-4c64-84e7-277e09187336" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 24 16:32:45.117437 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:45.117405    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:45.117905 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:45.117692    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:45.118450 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:45.118433    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:32:56.845793 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:56.845758    2572 generic.go:356] "Generic (PLEG): container finished" podID="0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" containerID="844bb2d2fdc3ee3336fda28387cd63c325c8f4403aaeecaf6264e312bc021f6c" exitCode=0
Jul 24 16:32:56.846239 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:56.845806    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2" event={"ID":"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4","Type":"ContainerDied","Data":"844bb2d2fdc3ee3336fda28387cd63c325c8f4403aaeecaf6264e312bc021f6c"}
Jul 24 16:32:56.846239 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:56.846222    2572 scope.go:117] "RemoveContainer" containerID="844bb2d2fdc3ee3336fda28387cd63c325c8f4403aaeecaf6264e312bc021f6c"
Jul 24 16:32:56.847303 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:56.847272    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:32:56.847407 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:56.847354    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:32:56.847407 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:56.847402    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 24 16:32:57.852683 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:57.852651    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2" event={"ID":"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4","Type":"ContainerStarted","Data":"17d035b74c6dc51f5749db153815bdfd2ff72b505c7e4e436df7b257f007b0be"}
Jul 24 16:32:58.787264 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:58.787222    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:32:58.787477 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:58.787456    2572 patch_prober.go:28] interesting pod/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 24 16:32:58.787524 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:58.787505    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2" podUID="0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 24 16:32:59.787393 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:59.787357    2572 patch_prober.go:28] interesting pod/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 24 16:32:59.787813 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:32:59.787415    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2" podUID="0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 24 16:33:00.786875 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:00.786831    2572 patch_prober.go:28] interesting pod/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 24 16:33:00.787090 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:00.786906    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2" podUID="0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 24 16:33:01.791149 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:01.791115    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:33:01.791567 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:01.791411    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:33:01.792190 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:01.792173    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:33:08.992396 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:08.992364    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"]
Jul 24 16:33:08.994834 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:08.994814    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:08.997604 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:08.997578    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-0107ee\"/\"kube-root-ca.crt\""
Jul 24 16:33:08.998795 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:08.998767    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-0107ee\"/\"openshift-service-ca.crt\""
Jul 24 16:33:08.998883 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:08.998819    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-0107ee\"/\"default-dockercfg-jd9gp\""
Jul 24 16:33:08.998883 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:08.998847    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-0107ee\"/\"ipp-route-0107ee-kserve-self-signed-certs\""
Jul 24 16:33:09.006718 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.006695    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"]
Jul 24 16:33:09.099110 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.099072    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-home\") pod \"ipp-route-0107ee-kserve-58bdb95947-d548p\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") " pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.099298 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.099125    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-tmp-dir\") pod \"ipp-route-0107ee-kserve-58bdb95947-d548p\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") " pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.099298 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.099211    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c52d81-6736-42c2-bb17-1522a24caa72-tls-certs\") pod \"ipp-route-0107ee-kserve-58bdb95947-d548p\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") " pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.099298 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.099238    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcxtc\" (UniqueName: \"kubernetes.io/projected/a4c52d81-6736-42c2-bb17-1522a24caa72-kube-api-access-bcxtc\") pod \"ipp-route-0107ee-kserve-58bdb95947-d548p\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") " pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.099477 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.099306    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-dshm\") pod \"ipp-route-0107ee-kserve-58bdb95947-d548p\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") " pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.099477 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.099336    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-model-cache\") pod \"ipp-route-0107ee-kserve-58bdb95947-d548p\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") " pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.200876 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.200840    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-home\") pod \"ipp-route-0107ee-kserve-58bdb95947-d548p\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") " pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.201098 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.200887    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-tmp-dir\") pod \"ipp-route-0107ee-kserve-58bdb95947-d548p\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") " pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.201098 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.200979    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c52d81-6736-42c2-bb17-1522a24caa72-tls-certs\") pod \"ipp-route-0107ee-kserve-58bdb95947-d548p\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") " pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.201098 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.200997    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcxtc\" (UniqueName: \"kubernetes.io/projected/a4c52d81-6736-42c2-bb17-1522a24caa72-kube-api-access-bcxtc\") pod \"ipp-route-0107ee-kserve-58bdb95947-d548p\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") " pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.201098 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.201043    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-dshm\") pod \"ipp-route-0107ee-kserve-58bdb95947-d548p\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") " pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.201098 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.201073    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-model-cache\") pod \"ipp-route-0107ee-kserve-58bdb95947-d548p\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") " pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.201381 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.201350    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-home\") pod \"ipp-route-0107ee-kserve-58bdb95947-d548p\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") " pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.201381 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.201372    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-tmp-dir\") pod \"ipp-route-0107ee-kserve-58bdb95947-d548p\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") " pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.201552 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.201516    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-model-cache\") pod \"ipp-route-0107ee-kserve-58bdb95947-d548p\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") " pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.203442 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.203422    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-dshm\") pod \"ipp-route-0107ee-kserve-58bdb95947-d548p\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") " pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.203743 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.203725    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c52d81-6736-42c2-bb17-1522a24caa72-tls-certs\") pod \"ipp-route-0107ee-kserve-58bdb95947-d548p\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") " pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.210807 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.210784    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcxtc\" (UniqueName: \"kubernetes.io/projected/a4c52d81-6736-42c2-bb17-1522a24caa72-kube-api-access-bcxtc\") pod \"ipp-route-0107ee-kserve-58bdb95947-d548p\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") " pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.305074 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.304973    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:09.447982 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.447953    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"]
Jul 24 16:33:09.450248 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:33:09.450212    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4c52d81_6736_42c2_bb17_1522a24caa72.slice/crio-2620bfcb3fd542e3c5271dd1166482fb093eb1e1c79717094cd0e4e9d77344c1 WatchSource:0}: Error finding container 2620bfcb3fd542e3c5271dd1166482fb093eb1e1c79717094cd0e4e9d77344c1: Status 404 returned error can't find the container with id 2620bfcb3fd542e3c5271dd1166482fb093eb1e1c79717094cd0e4e9d77344c1
Jul 24 16:33:09.913516 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.913477    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p" event={"ID":"a4c52d81-6736-42c2-bb17-1522a24caa72","Type":"ContainerStarted","Data":"0b8248d18fd2592fc84ee2bbf2ed2542f23b6e31b7843e96fe6ee68cf5972e13"}
Jul 24 16:33:09.913516 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.913525    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p" event={"ID":"a4c52d81-6736-42c2-bb17-1522a24caa72","Type":"ContainerStarted","Data":"2620bfcb3fd542e3c5271dd1166482fb093eb1e1c79717094cd0e4e9d77344c1"}
Jul 24 16:33:09.935560 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:09.935513    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p" podStartSLOduration=1.935499308 podStartE2EDuration="1.935499308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:33:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:33:09.933848757 +0000 UTC m=+2869.218176364" watchObservedRunningTime="2026-07-24 16:33:09.935499308 +0000 UTC m=+2869.219826957"
Jul 24 16:33:19.305441 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:19.305355    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:19.305441 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:19.305401    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:19.317937 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:19.317897    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:33:19.971004 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:33:19.970972    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:34:06.203163 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:06.203125    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw"]
Jul 24 16:34:06.205439 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:06.205421    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw"
Jul 24 16:34:06.208006 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:06.207984    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 16:34:06.208138 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:06.208079    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kv2w9\""
Jul 24 16:34:06.213975 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:06.213945    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw"]
Jul 24 16:34:06.326767 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:06.326729    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/536b4ca7-d1db-492e-ab9b-72f22d055a9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw\" (UID: \"536b4ca7-d1db-492e-ab9b-72f22d055a9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw"
Jul 24 16:34:06.427438 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:06.427394    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/536b4ca7-d1db-492e-ab9b-72f22d055a9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw\" (UID: \"536b4ca7-d1db-492e-ab9b-72f22d055a9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw"
Jul 24 16:34:06.428113 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:06.428092    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/536b4ca7-d1db-492e-ab9b-72f22d055a9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw\" (UID: \"536b4ca7-d1db-492e-ab9b-72f22d055a9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw"
Jul 24 16:34:06.517021 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:06.516942    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw"
Jul 24 16:34:06.651987 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:06.651961    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw"]
Jul 24 16:34:06.654496 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:34:06.654471    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod536b4ca7_d1db_492e_ab9b_72f22d055a9b.slice/crio-6bdcb0eda887eb74fe40f8612d7d34880d76355ed4180830f60094f8d5a6be8a WatchSource:0}: Error finding container 6bdcb0eda887eb74fe40f8612d7d34880d76355ed4180830f60094f8d5a6be8a: Status 404 returned error can't find the container with id 6bdcb0eda887eb74fe40f8612d7d34880d76355ed4180830f60094f8d5a6be8a
Jul 24 16:34:07.177338 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:07.177296    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw" event={"ID":"536b4ca7-d1db-492e-ab9b-72f22d055a9b","Type":"ContainerStarted","Data":"cd87a23eca1332c8f3ef99896a595187611349ab56de9ff9b7b436b8c9f33a37"}
Jul 24 16:34:07.177522 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:07.177359    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw" event={"ID":"536b4ca7-d1db-492e-ab9b-72f22d055a9b","Type":"ContainerStarted","Data":"6bdcb0eda887eb74fe40f8612d7d34880d76355ed4180830f60094f8d5a6be8a"}
Jul 24 16:34:07.206897 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:07.206850    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw" podStartSLOduration=1.206834892 podStartE2EDuration="1.206834892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:34:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:34:07.204501265 +0000 UTC m=+2926.488828872" watchObservedRunningTime="2026-07-24 16:34:07.206834892 +0000 UTC m=+2926.491162559"
Jul 24 16:34:08.183868 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:08.183830    2572 generic.go:356] "Generic (PLEG): container finished" podID="536b4ca7-d1db-492e-ab9b-72f22d055a9b" containerID="cd87a23eca1332c8f3ef99896a595187611349ab56de9ff9b7b436b8c9f33a37" exitCode=0
Jul 24 16:34:08.184084 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:08.183937    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw" event={"ID":"536b4ca7-d1db-492e-ab9b-72f22d055a9b","Type":"ContainerDied","Data":"cd87a23eca1332c8f3ef99896a595187611349ab56de9ff9b7b436b8c9f33a37"}
Jul 24 16:34:09.322896 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:09.322866    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw"
Jul 24 16:34:09.455566 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:09.455479    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/536b4ca7-d1db-492e-ab9b-72f22d055a9b-maas-api-service-ca\") pod \"536b4ca7-d1db-492e-ab9b-72f22d055a9b\" (UID: \"536b4ca7-d1db-492e-ab9b-72f22d055a9b\") "
Jul 24 16:34:09.455821 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:09.455799    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/536b4ca7-d1db-492e-ab9b-72f22d055a9b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "536b4ca7-d1db-492e-ab9b-72f22d055a9b" (UID: "536b4ca7-d1db-492e-ab9b-72f22d055a9b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:34:09.556669 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:09.556629    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/536b4ca7-d1db-492e-ab9b-72f22d055a9b-maas-api-service-ca\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:34:10.194896 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:10.194868    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw"
Jul 24 16:34:10.195099 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:10.194859    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw" event={"ID":"536b4ca7-d1db-492e-ab9b-72f22d055a9b","Type":"ContainerDied","Data":"6bdcb0eda887eb74fe40f8612d7d34880d76355ed4180830f60094f8d5a6be8a"}
Jul 24 16:34:10.195099 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:10.194971    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6bdcb0eda887eb74fe40f8612d7d34880d76355ed4180830f60094f8d5a6be8a"
Jul 24 16:34:21.288611 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:21.288564    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw"]
Jul 24 16:34:21.290524 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:21.290489    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2ba900-qvwmw"]
Jul 24 16:34:23.222867 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:23.222832    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="536b4ca7-d1db-492e-ab9b-72f22d055a9b" path="/var/lib/kubelet/pods/536b4ca7-d1db-492e-ab9b-72f22d055a9b/volumes"
Jul 24 16:34:57.122980 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:34:57.122882    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"]
Jul 24 16:35:12.142468 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.142418    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p" podUID="a4c52d81-6736-42c2-bb17-1522a24caa72" containerName="main" containerID="cri-o://0b8248d18fd2592fc84ee2bbf2ed2542f23b6e31b7843e96fe6ee68cf5972e13" gracePeriod=45
Jul 24 16:35:12.395651 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.395627    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:35:12.442969 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.442906    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-dshm\") pod \"a4c52d81-6736-42c2-bb17-1522a24caa72\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") "
Jul 24 16:35:12.443149 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.443001    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcxtc\" (UniqueName: \"kubernetes.io/projected/a4c52d81-6736-42c2-bb17-1522a24caa72-kube-api-access-bcxtc\") pod \"a4c52d81-6736-42c2-bb17-1522a24caa72\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") "
Jul 24 16:35:12.443149 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.443047    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-home\") pod \"a4c52d81-6736-42c2-bb17-1522a24caa72\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") "
Jul 24 16:35:12.443149 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.443073    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-model-cache\") pod \"a4c52d81-6736-42c2-bb17-1522a24caa72\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") "
Jul 24 16:35:12.443149 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.443104    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-tmp-dir\") pod \"a4c52d81-6736-42c2-bb17-1522a24caa72\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") "
Jul 24 16:35:12.443149 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.443134    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c52d81-6736-42c2-bb17-1522a24caa72-tls-certs\") pod \"a4c52d81-6736-42c2-bb17-1522a24caa72\" (UID: \"a4c52d81-6736-42c2-bb17-1522a24caa72\") "
Jul 24 16:35:12.443436 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.443249    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-home" (OuterVolumeSpecName: "home") pod "a4c52d81-6736-42c2-bb17-1522a24caa72" (UID: "a4c52d81-6736-42c2-bb17-1522a24caa72"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:35:12.443536 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.443505    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-home\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:35:12.443536 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.443511    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a4c52d81-6736-42c2-bb17-1522a24caa72" (UID: "a4c52d81-6736-42c2-bb17-1522a24caa72"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:35:12.443650 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.443605    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-model-cache" (OuterVolumeSpecName: "model-cache") pod "a4c52d81-6736-42c2-bb17-1522a24caa72" (UID: "a4c52d81-6736-42c2-bb17-1522a24caa72"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:35:12.445600 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.445569    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4c52d81-6736-42c2-bb17-1522a24caa72-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a4c52d81-6736-42c2-bb17-1522a24caa72" (UID: "a4c52d81-6736-42c2-bb17-1522a24caa72"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 24 16:35:12.445600 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.445574    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4c52d81-6736-42c2-bb17-1522a24caa72-kube-api-access-bcxtc" (OuterVolumeSpecName: "kube-api-access-bcxtc") pod "a4c52d81-6736-42c2-bb17-1522a24caa72" (UID: "a4c52d81-6736-42c2-bb17-1522a24caa72"). InnerVolumeSpecName "kube-api-access-bcxtc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:35:12.445795 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.445565    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-dshm" (OuterVolumeSpecName: "dshm") pod "a4c52d81-6736-42c2-bb17-1522a24caa72" (UID: "a4c52d81-6736-42c2-bb17-1522a24caa72"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:35:12.496006 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.495968    2572 generic.go:356] "Generic (PLEG): container finished" podID="a4c52d81-6736-42c2-bb17-1522a24caa72" containerID="0b8248d18fd2592fc84ee2bbf2ed2542f23b6e31b7843e96fe6ee68cf5972e13" exitCode=0
Jul 24 16:35:12.496006 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.496010    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"
Jul 24 16:35:12.496220 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.496027    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p" event={"ID":"a4c52d81-6736-42c2-bb17-1522a24caa72","Type":"ContainerDied","Data":"0b8248d18fd2592fc84ee2bbf2ed2542f23b6e31b7843e96fe6ee68cf5972e13"}
Jul 24 16:35:12.496220 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.496059    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p" event={"ID":"a4c52d81-6736-42c2-bb17-1522a24caa72","Type":"ContainerDied","Data":"2620bfcb3fd542e3c5271dd1166482fb093eb1e1c79717094cd0e4e9d77344c1"}
Jul 24 16:35:12.496220 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.496075    2572 scope.go:117] "RemoveContainer" containerID="0b8248d18fd2592fc84ee2bbf2ed2542f23b6e31b7843e96fe6ee68cf5972e13"
Jul 24 16:35:12.505523 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.505503    2572 scope.go:117] "RemoveContainer" containerID="0b8248d18fd2592fc84ee2bbf2ed2542f23b6e31b7843e96fe6ee68cf5972e13"
Jul 24 16:35:12.505799 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:35:12.505780    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b8248d18fd2592fc84ee2bbf2ed2542f23b6e31b7843e96fe6ee68cf5972e13\": container with ID starting with 0b8248d18fd2592fc84ee2bbf2ed2542f23b6e31b7843e96fe6ee68cf5972e13 not found: ID does not exist" containerID="0b8248d18fd2592fc84ee2bbf2ed2542f23b6e31b7843e96fe6ee68cf5972e13"
Jul 24 16:35:12.505870 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.505807    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b8248d18fd2592fc84ee2bbf2ed2542f23b6e31b7843e96fe6ee68cf5972e13"} err="failed to get container status \"0b8248d18fd2592fc84ee2bbf2ed2542f23b6e31b7843e96fe6ee68cf5972e13\": rpc error: code = NotFound desc = could not find container \"0b8248d18fd2592fc84ee2bbf2ed2542f23b6e31b7843e96fe6ee68cf5972e13\": container with ID starting with 0b8248d18fd2592fc84ee2bbf2ed2542f23b6e31b7843e96fe6ee68cf5972e13 not found: ID does not exist"
Jul 24 16:35:12.517363 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.517334    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"]
Jul 24 16:35:12.523045 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.523014    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-0107ee/ipp-route-0107ee-kserve-58bdb95947-d548p"]
Jul 24 16:35:12.544640 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.544610    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-dshm\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:35:12.544640 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.544638    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcxtc\" (UniqueName: \"kubernetes.io/projected/a4c52d81-6736-42c2-bb17-1522a24caa72-kube-api-access-bcxtc\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:35:12.544824 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.544650    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-model-cache\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:35:12.544824 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.544660    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4c52d81-6736-42c2-bb17-1522a24caa72-tmp-dir\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:35:12.544824 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:12.544671    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a4c52d81-6736-42c2-bb17-1522a24caa72-tls-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:35:13.223479 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:13.223447    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4c52d81-6736-42c2-bb17-1522a24caa72" path="/var/lib/kubelet/pods/a4c52d81-6736-42c2-bb17-1522a24caa72/volumes"
Jul 24 16:35:21.324555 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:21.324525    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 16:35:21.332942 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:21.332897    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-lj8js_0e30eb70-1243-4f4b-8a15-5df72a224f51/ovn-acl-logging/0.log"
Jul 24 16:35:45.921378 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:45.921339    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"]
Jul 24 16:35:45.922013 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:45.921602    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n" podUID="cdf7137c-282c-4c64-84e7-277e09187336" containerName="istio-proxy" containerID="cri-o://fc05666100a5204ff2e2dc0aadc65c9793ba8afd075cddcf2f8cb275c6a3f516" gracePeriod=30
Jul 24 16:35:51.181323 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.181294    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:35:51.195798 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.195710    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-workload-socket\") pod \"cdf7137c-282c-4c64-84e7-277e09187336\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") "
Jul 24 16:35:51.195798 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.195747    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-workload-certs\") pod \"cdf7137c-282c-4c64-84e7-277e09187336\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") "
Jul 24 16:35:51.195798 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.195765    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdf7137c-282c-4c64-84e7-277e09187336-istiod-ca-cert\") pod \"cdf7137c-282c-4c64-84e7-277e09187336\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") "
Jul 24 16:35:51.195798 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.195782    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plpqh\" (UniqueName: \"kubernetes.io/projected/cdf7137c-282c-4c64-84e7-277e09187336-kube-api-access-plpqh\") pod \"cdf7137c-282c-4c64-84e7-277e09187336\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") "
Jul 24 16:35:51.196159 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.195830    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdf7137c-282c-4c64-84e7-277e09187336-istio-podinfo\") pod \"cdf7137c-282c-4c64-84e7-277e09187336\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") "
Jul 24 16:35:51.196159 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.195850    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-istio-data\") pod \"cdf7137c-282c-4c64-84e7-277e09187336\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") "
Jul 24 16:35:51.196159 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.195876    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdf7137c-282c-4c64-84e7-277e09187336-istio-token\") pod \"cdf7137c-282c-4c64-84e7-277e09187336\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") "
Jul 24 16:35:51.196159 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.195890    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-credential-socket\") pod \"cdf7137c-282c-4c64-84e7-277e09187336\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") "
Jul 24 16:35:51.196159 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.195951    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-istio-envoy\") pod \"cdf7137c-282c-4c64-84e7-277e09187336\" (UID: \"cdf7137c-282c-4c64-84e7-277e09187336\") "
Jul 24 16:35:51.196159 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.196111    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cdf7137c-282c-4c64-84e7-277e09187336" (UID: "cdf7137c-282c-4c64-84e7-277e09187336"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:35:51.196485 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.196244    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-workload-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:35:51.196485 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.196285    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cdf7137c-282c-4c64-84e7-277e09187336" (UID: "cdf7137c-282c-4c64-84e7-277e09187336"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:35:51.196485 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.196217    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdf7137c-282c-4c64-84e7-277e09187336-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cdf7137c-282c-4c64-84e7-277e09187336" (UID: "cdf7137c-282c-4c64-84e7-277e09187336"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:35:51.196485 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.196450    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cdf7137c-282c-4c64-84e7-277e09187336" (UID: "cdf7137c-282c-4c64-84e7-277e09187336"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:35:51.197182 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.197153    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-istio-data" (OuterVolumeSpecName: "istio-data") pod "cdf7137c-282c-4c64-84e7-277e09187336" (UID: "cdf7137c-282c-4c64-84e7-277e09187336"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:35:51.198314 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.198287    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cdf7137c-282c-4c64-84e7-277e09187336" (UID: "cdf7137c-282c-4c64-84e7-277e09187336"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:35:51.198314 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.198299    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdf7137c-282c-4c64-84e7-277e09187336-kube-api-access-plpqh" (OuterVolumeSpecName: "kube-api-access-plpqh") pod "cdf7137c-282c-4c64-84e7-277e09187336" (UID: "cdf7137c-282c-4c64-84e7-277e09187336"). InnerVolumeSpecName "kube-api-access-plpqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:35:51.198840 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.198814    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cdf7137c-282c-4c64-84e7-277e09187336-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cdf7137c-282c-4c64-84e7-277e09187336" (UID: "cdf7137c-282c-4c64-84e7-277e09187336"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 16:35:51.198840 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.198829    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdf7137c-282c-4c64-84e7-277e09187336-istio-token" (OuterVolumeSpecName: "istio-token") pod "cdf7137c-282c-4c64-84e7-277e09187336" (UID: "cdf7137c-282c-4c64-84e7-277e09187336"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:35:51.297500 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.297454    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-workload-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:35:51.297500 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.297492    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cdf7137c-282c-4c64-84e7-277e09187336-istiod-ca-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:35:51.297500 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.297501    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plpqh\" (UniqueName: \"kubernetes.io/projected/cdf7137c-282c-4c64-84e7-277e09187336-kube-api-access-plpqh\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:35:51.297500 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.297511    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cdf7137c-282c-4c64-84e7-277e09187336-istio-podinfo\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:35:51.297793 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.297520    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-istio-data\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:35:51.297793 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.297528    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cdf7137c-282c-4c64-84e7-277e09187336-istio-token\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:35:51.297793 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.297535    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-credential-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:35:51.297793 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.297543    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cdf7137c-282c-4c64-84e7-277e09187336-istio-envoy\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:35:51.686297 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.686258    2572 generic.go:356] "Generic (PLEG): container finished" podID="cdf7137c-282c-4c64-84e7-277e09187336" containerID="fc05666100a5204ff2e2dc0aadc65c9793ba8afd075cddcf2f8cb275c6a3f516" exitCode=0
Jul 24 16:35:51.686508 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.686330    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"
Jul 24 16:35:51.686508 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.686347    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n" event={"ID":"cdf7137c-282c-4c64-84e7-277e09187336","Type":"ContainerDied","Data":"fc05666100a5204ff2e2dc0aadc65c9793ba8afd075cddcf2f8cb275c6a3f516"}
Jul 24 16:35:51.686508 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.686412    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n" event={"ID":"cdf7137c-282c-4c64-84e7-277e09187336","Type":"ContainerDied","Data":"9056f5ed44ccc0dc5210af103876da49d58a4a0d0c2cd9ac68ab8743557baeed"}
Jul 24 16:35:51.686508 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.686443    2572 scope.go:117] "RemoveContainer" containerID="fc05666100a5204ff2e2dc0aadc65c9793ba8afd075cddcf2f8cb275c6a3f516"
Jul 24 16:35:51.695091 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.695067    2572 scope.go:117] "RemoveContainer" containerID="f06949d163691aa7893e4895fd5e8424ebadc936844b1706fb4bc9841c19cdeb"
Jul 24 16:35:51.703501 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.703482    2572 scope.go:117] "RemoveContainer" containerID="fc05666100a5204ff2e2dc0aadc65c9793ba8afd075cddcf2f8cb275c6a3f516"
Jul 24 16:35:51.703809 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:35:51.703786    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc05666100a5204ff2e2dc0aadc65c9793ba8afd075cddcf2f8cb275c6a3f516\": container with ID starting with fc05666100a5204ff2e2dc0aadc65c9793ba8afd075cddcf2f8cb275c6a3f516 not found: ID does not exist" containerID="fc05666100a5204ff2e2dc0aadc65c9793ba8afd075cddcf2f8cb275c6a3f516"
Jul 24 16:35:51.703893 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.703823    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc05666100a5204ff2e2dc0aadc65c9793ba8afd075cddcf2f8cb275c6a3f516"} err="failed to get container status \"fc05666100a5204ff2e2dc0aadc65c9793ba8afd075cddcf2f8cb275c6a3f516\": rpc error: code = NotFound desc = could not find container \"fc05666100a5204ff2e2dc0aadc65c9793ba8afd075cddcf2f8cb275c6a3f516\": container with ID starting with fc05666100a5204ff2e2dc0aadc65c9793ba8afd075cddcf2f8cb275c6a3f516 not found: ID does not exist"
Jul 24 16:35:51.703893 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.703848    2572 scope.go:117] "RemoveContainer" containerID="f06949d163691aa7893e4895fd5e8424ebadc936844b1706fb4bc9841c19cdeb"
Jul 24 16:35:51.704143 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:35:51.704123    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f06949d163691aa7893e4895fd5e8424ebadc936844b1706fb4bc9841c19cdeb\": container with ID starting with f06949d163691aa7893e4895fd5e8424ebadc936844b1706fb4bc9841c19cdeb not found: ID does not exist" containerID="f06949d163691aa7893e4895fd5e8424ebadc936844b1706fb4bc9841c19cdeb"
Jul 24 16:35:51.704191 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.704150    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f06949d163691aa7893e4895fd5e8424ebadc936844b1706fb4bc9841c19cdeb"} err="failed to get container status \"f06949d163691aa7893e4895fd5e8424ebadc936844b1706fb4bc9841c19cdeb\": rpc error: code = NotFound desc = could not find container \"f06949d163691aa7893e4895fd5e8424ebadc936844b1706fb4bc9841c19cdeb\": container with ID starting with f06949d163691aa7893e4895fd5e8424ebadc936844b1706fb4bc9841c19cdeb not found: ID does not exist"
Jul 24 16:35:51.707389 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.707365    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"]
Jul 24 16:35:51.711269 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:51.711244    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-0107ee-openshift-default-5c8c8b99f4-dzz8n"]
Jul 24 16:35:53.222784 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:35:53.222747    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdf7137c-282c-4c64-84e7-277e09187336" path="/var/lib/kubelet/pods/cdf7137c-282c-4c64-84e7-277e09187336/volumes"
Jul 24 16:36:08.312600 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.312562    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"]
Jul 24 16:36:08.313184 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.312837    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2" podUID="0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" containerName="istio-proxy" containerID="cri-o://17d035b74c6dc51f5749db153815bdfd2ff72b505c7e4e436df7b257f007b0be" gracePeriod=30
Jul 24 16:36:08.713278 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.713243    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb"]
Jul 24 16:36:08.713584 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.713571    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdf7137c-282c-4c64-84e7-277e09187336" containerName="istio-proxy"
Jul 24 16:36:08.713632 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.713585    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdf7137c-282c-4c64-84e7-277e09187336" containerName="istio-proxy"
Jul 24 16:36:08.713632 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.713598    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4c52d81-6736-42c2-bb17-1522a24caa72" containerName="main"
Jul 24 16:36:08.713632 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.713604    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4c52d81-6736-42c2-bb17-1522a24caa72" containerName="main"
Jul 24 16:36:08.713632 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.713619    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="536b4ca7-d1db-492e-ab9b-72f22d055a9b" containerName="revoke-keys"
Jul 24 16:36:08.713632 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.713625    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="536b4ca7-d1db-492e-ab9b-72f22d055a9b" containerName="revoke-keys"
Jul 24 16:36:08.713791 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.713690    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdf7137c-282c-4c64-84e7-277e09187336" containerName="istio-proxy"
Jul 24 16:36:08.713791 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.713700    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdf7137c-282c-4c64-84e7-277e09187336" containerName="istio-proxy"
Jul 24 16:36:08.713791 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.713707    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4c52d81-6736-42c2-bb17-1522a24caa72" containerName="main"
Jul 24 16:36:08.713791 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.713714    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="536b4ca7-d1db-492e-ab9b-72f22d055a9b" containerName="revoke-keys"
Jul 24 16:36:08.716814 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.716794    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb"
Jul 24 16:36:08.719390 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.719365    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 24 16:36:08.719501 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.719369    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-kv2w9\""
Jul 24 16:36:08.725332 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.725301    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb"]
Jul 24 16:36:08.859319 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.859277    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73ea8d88-5704-42d7-a42f-19504b9c2f32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb\" (UID: \"73ea8d88-5704-42d7-a42f-19504b9c2f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb"
Jul 24 16:36:08.960724 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.960687    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73ea8d88-5704-42d7-a42f-19504b9c2f32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb\" (UID: \"73ea8d88-5704-42d7-a42f-19504b9c2f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb"
Jul 24 16:36:08.961359 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:08.961338    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73ea8d88-5704-42d7-a42f-19504b9c2f32-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb\" (UID: \"73ea8d88-5704-42d7-a42f-19504b9c2f32\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb"
Jul 24 16:36:09.028120 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:09.028023    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb"
Jul 24 16:36:09.168413 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:09.168386    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb"]
Jul 24 16:36:09.171019 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:36:09.170987    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73ea8d88_5704_42d7_a42f_19504b9c2f32.slice/crio-f455b6e9861d163c366f3292f0a9a5a1d0a518ad0d1e2cc9f22cbbcabe99a86f WatchSource:0}: Error finding container f455b6e9861d163c366f3292f0a9a5a1d0a518ad0d1e2cc9f22cbbcabe99a86f: Status 404 returned error can't find the container with id f455b6e9861d163c366f3292f0a9a5a1d0a518ad0d1e2cc9f22cbbcabe99a86f
Jul 24 16:36:09.772832 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:09.772740    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb" event={"ID":"73ea8d88-5704-42d7-a42f-19504b9c2f32","Type":"ContainerStarted","Data":"75f13c311dd69bd1578b8ed76c219a8dbb3c9c0bdcbccc5852fb0ffa0860d68b"}
Jul 24 16:36:09.772832 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:09.772798    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb" event={"ID":"73ea8d88-5704-42d7-a42f-19504b9c2f32","Type":"ContainerStarted","Data":"f455b6e9861d163c366f3292f0a9a5a1d0a518ad0d1e2cc9f22cbbcabe99a86f"}
Jul 24 16:36:09.789907 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:09.789851    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb" podStartSLOduration=1.789834249 podStartE2EDuration="1.789834249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:36:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:36:09.78829056 +0000 UTC m=+3049.072618168" watchObservedRunningTime="2026-07-24 16:36:09.789834249 +0000 UTC m=+3049.074161886"
Jul 24 16:36:10.779243 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:10.779208    2572 generic.go:356] "Generic (PLEG): container finished" podID="73ea8d88-5704-42d7-a42f-19504b9c2f32" containerID="75f13c311dd69bd1578b8ed76c219a8dbb3c9c0bdcbccc5852fb0ffa0860d68b" exitCode=0
Jul 24 16:36:10.779668 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:10.779289    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb" event={"ID":"73ea8d88-5704-42d7-a42f-19504b9c2f32","Type":"ContainerDied","Data":"75f13c311dd69bd1578b8ed76c219a8dbb3c9c0bdcbccc5852fb0ffa0860d68b"}
Jul 24 16:36:11.908001 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:11.907967    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb"
Jul 24 16:36:11.989518 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:11.989484    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73ea8d88-5704-42d7-a42f-19504b9c2f32-maas-api-service-ca\") pod \"73ea8d88-5704-42d7-a42f-19504b9c2f32\" (UID: \"73ea8d88-5704-42d7-a42f-19504b9c2f32\") "
Jul 24 16:36:11.989838 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:11.989815    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73ea8d88-5704-42d7-a42f-19504b9c2f32-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "73ea8d88-5704-42d7-a42f-19504b9c2f32" (UID: "73ea8d88-5704-42d7-a42f-19504b9c2f32"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:36:12.090380 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:12.090286    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73ea8d88-5704-42d7-a42f-19504b9c2f32-maas-api-service-ca\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:12.788904 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:12.788858    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb" event={"ID":"73ea8d88-5704-42d7-a42f-19504b9c2f32","Type":"ContainerDied","Data":"f455b6e9861d163c366f3292f0a9a5a1d0a518ad0d1e2cc9f22cbbcabe99a86f"}
Jul 24 16:36:12.789103 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:12.788931    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f455b6e9861d163c366f3292f0a9a5a1d0a518ad0d1e2cc9f22cbbcabe99a86f"
Jul 24 16:36:12.789103 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:12.788881    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb"
Jul 24 16:36:13.568914 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.568890    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:36:13.705105 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.705076    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-credential-socket\") pod \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") "
Jul 24 16:36:13.705377 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.705121    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-envoy\") pod \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") "
Jul 24 16:36:13.705377 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.705179    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-data\") pod \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") "
Jul 24 16:36:13.705377 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.705213    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istiod-ca-cert\") pod \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") "
Jul 24 16:36:13.705377 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.705261    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-podinfo\") pod \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") "
Jul 24 16:36:13.705377 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.705292    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zzlk\" (UniqueName: \"kubernetes.io/projected/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-kube-api-access-5zzlk\") pod \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") "
Jul 24 16:36:13.705377 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.705326    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-workload-socket\") pod \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") "
Jul 24 16:36:13.705377 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.705379    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-token\") pod \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") "
Jul 24 16:36:13.705766 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.705384    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" (UID: "0cb5ab5d-6d19-436a-8119-f5d2bce30bb4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:13.705766 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.705403    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-workload-certs\") pod \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\" (UID: \"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4\") "
Jul 24 16:36:13.705766 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.705696    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" (UID: "0cb5ab5d-6d19-436a-8119-f5d2bce30bb4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:13.705766 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.705696    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" (UID: "0cb5ab5d-6d19-436a-8119-f5d2bce30bb4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:36:13.705766 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.705700    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-credential-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:13.706239 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.705888    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" (UID: "0cb5ab5d-6d19-436a-8119-f5d2bce30bb4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:13.706239 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.706139    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-data" (OuterVolumeSpecName: "istio-data") pod "0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" (UID: "0cb5ab5d-6d19-436a-8119-f5d2bce30bb4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:13.707667 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.707642    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" (UID: "0cb5ab5d-6d19-436a-8119-f5d2bce30bb4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:13.707780 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.707730    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-kube-api-access-5zzlk" (OuterVolumeSpecName: "kube-api-access-5zzlk") pod "0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" (UID: "0cb5ab5d-6d19-436a-8119-f5d2bce30bb4"). InnerVolumeSpecName "kube-api-access-5zzlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:36:13.708279 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.708250    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-token" (OuterVolumeSpecName: "istio-token") pod "0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" (UID: "0cb5ab5d-6d19-436a-8119-f5d2bce30bb4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:36:13.708391 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.708285    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" (UID: "0cb5ab5d-6d19-436a-8119-f5d2bce30bb4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 16:36:13.796392 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.796351    2572 generic.go:356] "Generic (PLEG): container finished" podID="0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" containerID="17d035b74c6dc51f5749db153815bdfd2ff72b505c7e4e436df7b257f007b0be" exitCode=0
Jul 24 16:36:13.796582 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.796400    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"
Jul 24 16:36:13.796582 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.796407    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2" event={"ID":"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4","Type":"ContainerDied","Data":"17d035b74c6dc51f5749db153815bdfd2ff72b505c7e4e436df7b257f007b0be"}
Jul 24 16:36:13.796582 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.796458    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2" event={"ID":"0cb5ab5d-6d19-436a-8119-f5d2bce30bb4","Type":"ContainerDied","Data":"5416005293f410abb1bb594d35008ed9e27018ac456c76cd926b31def5955996"}
Jul 24 16:36:13.796582 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.796481    2572 scope.go:117] "RemoveContainer" containerID="17d035b74c6dc51f5749db153815bdfd2ff72b505c7e4e436df7b257f007b0be"
Jul 24 16:36:13.806540 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.806515    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istiod-ca-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:13.806540 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.806539    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-podinfo\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:13.806691 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.806548    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zzlk\" (UniqueName: \"kubernetes.io/projected/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-kube-api-access-5zzlk\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:13.806691 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.806559    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-workload-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:13.806691 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.806567    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-token\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:13.806691 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.806575    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-workload-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:13.806691 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.806582    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-envoy\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:13.806691 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.806591    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4-istio-data\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:13.806979 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.806963    2572 scope.go:117] "RemoveContainer" containerID="844bb2d2fdc3ee3336fda28387cd63c325c8f4403aaeecaf6264e312bc021f6c"
Jul 24 16:36:13.815110 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.815092    2572 scope.go:117] "RemoveContainer" containerID="17d035b74c6dc51f5749db153815bdfd2ff72b505c7e4e436df7b257f007b0be"
Jul 24 16:36:13.815384 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:36:13.815364    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17d035b74c6dc51f5749db153815bdfd2ff72b505c7e4e436df7b257f007b0be\": container with ID starting with 17d035b74c6dc51f5749db153815bdfd2ff72b505c7e4e436df7b257f007b0be not found: ID does not exist" containerID="17d035b74c6dc51f5749db153815bdfd2ff72b505c7e4e436df7b257f007b0be"
Jul 24 16:36:13.815433 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.815393    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17d035b74c6dc51f5749db153815bdfd2ff72b505c7e4e436df7b257f007b0be"} err="failed to get container status \"17d035b74c6dc51f5749db153815bdfd2ff72b505c7e4e436df7b257f007b0be\": rpc error: code = NotFound desc = could not find container \"17d035b74c6dc51f5749db153815bdfd2ff72b505c7e4e436df7b257f007b0be\": container with ID starting with 17d035b74c6dc51f5749db153815bdfd2ff72b505c7e4e436df7b257f007b0be not found: ID does not exist"
Jul 24 16:36:13.815433 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.815410    2572 scope.go:117] "RemoveContainer" containerID="844bb2d2fdc3ee3336fda28387cd63c325c8f4403aaeecaf6264e312bc021f6c"
Jul 24 16:36:13.815653 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:36:13.815636    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"844bb2d2fdc3ee3336fda28387cd63c325c8f4403aaeecaf6264e312bc021f6c\": container with ID starting with 844bb2d2fdc3ee3336fda28387cd63c325c8f4403aaeecaf6264e312bc021f6c not found: ID does not exist" containerID="844bb2d2fdc3ee3336fda28387cd63c325c8f4403aaeecaf6264e312bc021f6c"
Jul 24 16:36:13.815714 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.815662    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"844bb2d2fdc3ee3336fda28387cd63c325c8f4403aaeecaf6264e312bc021f6c"} err="failed to get container status \"844bb2d2fdc3ee3336fda28387cd63c325c8f4403aaeecaf6264e312bc021f6c\": rpc error: code = NotFound desc = could not find container \"844bb2d2fdc3ee3336fda28387cd63c325c8f4403aaeecaf6264e312bc021f6c\": container with ID starting with 844bb2d2fdc3ee3336fda28387cd63c325c8f4403aaeecaf6264e312bc021f6c not found: ID does not exist"
Jul 24 16:36:13.823037 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.823007    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"]
Jul 24 16:36:13.827224 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:13.827203    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-52d140-openshift-default-8448d6787b-fxmd2"]
Jul 24 16:36:15.224265 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:15.224230    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" path="/var/lib/kubelet/pods/0cb5ab5d-6d19-436a-8119-f5d2bce30bb4/volumes"
Jul 24 16:36:23.787271 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:23.787232    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb"]
Jul 24 16:36:23.790231 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:23.790204    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-764b09-lxwhb"]
Jul 24 16:36:25.223530 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:25.223487    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73ea8d88-5704-42d7-a42f-19504b9c2f32" path="/var/lib/kubelet/pods/73ea8d88-5704-42d7-a42f-19504b9c2f32/volumes"
Jul 24 16:36:30.327719 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:30.327686    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"]
Jul 24 16:36:30.328241 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:30.327939    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b" podUID="55e3284a-cf56-4f90-bb18-d3ee1d754149" containerName="istio-proxy" containerID="cri-o://2e4ae12584ff08fbdc3511288c31afb36e597f39c42995b993a70211dd872fb7" gracePeriod=30
Jul 24 16:36:31.021197 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:31.021150    2572 patch_prober.go:28] interesting pod/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 24 16:36:31.021364 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:31.021242    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b" podUID="55e3284a-cf56-4f90-bb18-d3ee1d754149" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 24 16:36:35.577147 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.577120    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:36:35.601877 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.601801    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-podinfo\") pod \"55e3284a-cf56-4f90-bb18-d3ee1d754149\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") "
Jul 24 16:36:35.601877 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.601835    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-credential-socket\") pod \"55e3284a-cf56-4f90-bb18-d3ee1d754149\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") "
Jul 24 16:36:35.601877 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.601860    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-data\") pod \"55e3284a-cf56-4f90-bb18-d3ee1d754149\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") "
Jul 24 16:36:35.609661 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.602383    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "55e3284a-cf56-4f90-bb18-d3ee1d754149" (UID: "55e3284a-cf56-4f90-bb18-d3ee1d754149"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:35.609661 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.603238    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-data" (OuterVolumeSpecName: "istio-data") pod "55e3284a-cf56-4f90-bb18-d3ee1d754149" (UID: "55e3284a-cf56-4f90-bb18-d3ee1d754149"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:35.609943 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.609692    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk5gj\" (UniqueName: \"kubernetes.io/projected/55e3284a-cf56-4f90-bb18-d3ee1d754149-kube-api-access-sk5gj\") pod \"55e3284a-cf56-4f90-bb18-d3ee1d754149\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") "
Jul 24 16:36:35.610022 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.609965    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-envoy\") pod \"55e3284a-cf56-4f90-bb18-d3ee1d754149\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") "
Jul 24 16:36:35.610022 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.610010    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-workload-socket\") pod \"55e3284a-cf56-4f90-bb18-d3ee1d754149\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") "
Jul 24 16:36:35.610143 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.610053    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-workload-certs\") pod \"55e3284a-cf56-4f90-bb18-d3ee1d754149\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") "
Jul 24 16:36:35.610143 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.610121    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-token\") pod \"55e3284a-cf56-4f90-bb18-d3ee1d754149\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") "
Jul 24 16:36:35.610251 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.610147    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55e3284a-cf56-4f90-bb18-d3ee1d754149-istiod-ca-cert\") pod \"55e3284a-cf56-4f90-bb18-d3ee1d754149\" (UID: \"55e3284a-cf56-4f90-bb18-d3ee1d754149\") "
Jul 24 16:36:35.610392 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.610364    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "55e3284a-cf56-4f90-bb18-d3ee1d754149" (UID: "55e3284a-cf56-4f90-bb18-d3ee1d754149"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 16:36:35.610649 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.610626    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "55e3284a-cf56-4f90-bb18-d3ee1d754149" (UID: "55e3284a-cf56-4f90-bb18-d3ee1d754149"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:35.610755 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.610679    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-podinfo\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:35.610755 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.610701    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-credential-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:35.610755 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.610716    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-data\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:35.611018 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.610993    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "55e3284a-cf56-4f90-bb18-d3ee1d754149" (UID: "55e3284a-cf56-4f90-bb18-d3ee1d754149"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:35.611502 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.611473    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55e3284a-cf56-4f90-bb18-d3ee1d754149-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "55e3284a-cf56-4f90-bb18-d3ee1d754149" (UID: "55e3284a-cf56-4f90-bb18-d3ee1d754149"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:36:35.612692 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.612644    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "55e3284a-cf56-4f90-bb18-d3ee1d754149" (UID: "55e3284a-cf56-4f90-bb18-d3ee1d754149"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:35.612780 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.612747    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55e3284a-cf56-4f90-bb18-d3ee1d754149-kube-api-access-sk5gj" (OuterVolumeSpecName: "kube-api-access-sk5gj") pod "55e3284a-cf56-4f90-bb18-d3ee1d754149" (UID: "55e3284a-cf56-4f90-bb18-d3ee1d754149"). InnerVolumeSpecName "kube-api-access-sk5gj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:36:35.613418 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.613385    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-token" (OuterVolumeSpecName: "istio-token") pod "55e3284a-cf56-4f90-bb18-d3ee1d754149" (UID: "55e3284a-cf56-4f90-bb18-d3ee1d754149"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:36:35.712018 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.711979    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk5gj\" (UniqueName: \"kubernetes.io/projected/55e3284a-cf56-4f90-bb18-d3ee1d754149-kube-api-access-sk5gj\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:35.712018 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.712016    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-envoy\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:35.712018 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.712026    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-workload-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:35.712337 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.712035    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55e3284a-cf56-4f90-bb18-d3ee1d754149-workload-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:35.712337 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.712044    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55e3284a-cf56-4f90-bb18-d3ee1d754149-istio-token\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:35.712337 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.712052    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55e3284a-cf56-4f90-bb18-d3ee1d754149-istiod-ca-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:35.907124 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.907082    2572 generic.go:356] "Generic (PLEG): container finished" podID="55e3284a-cf56-4f90-bb18-d3ee1d754149" containerID="2e4ae12584ff08fbdc3511288c31afb36e597f39c42995b993a70211dd872fb7" exitCode=0
Jul 24 16:36:35.907307 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.907153    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"
Jul 24 16:36:35.907307 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.907166    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b" event={"ID":"55e3284a-cf56-4f90-bb18-d3ee1d754149","Type":"ContainerDied","Data":"2e4ae12584ff08fbdc3511288c31afb36e597f39c42995b993a70211dd872fb7"}
Jul 24 16:36:35.907307 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.907257    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b" event={"ID":"55e3284a-cf56-4f90-bb18-d3ee1d754149","Type":"ContainerDied","Data":"8503f71224ecd22df5d9602adae4790fed3ea4a1d28d9ee51df62b0d27f4773b"}
Jul 24 16:36:35.907307 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.907280    2572 scope.go:117] "RemoveContainer" containerID="2e4ae12584ff08fbdc3511288c31afb36e597f39c42995b993a70211dd872fb7"
Jul 24 16:36:35.917236 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.917216    2572 scope.go:117] "RemoveContainer" containerID="e2629b6be0f608bac59a0932ec4dad2da6fa00ecb338eef8632c14150ce50f76"
Jul 24 16:36:35.925549 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.925529    2572 scope.go:117] "RemoveContainer" containerID="2e4ae12584ff08fbdc3511288c31afb36e597f39c42995b993a70211dd872fb7"
Jul 24 16:36:35.925810 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:36:35.925790    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e4ae12584ff08fbdc3511288c31afb36e597f39c42995b993a70211dd872fb7\": container with ID starting with 2e4ae12584ff08fbdc3511288c31afb36e597f39c42995b993a70211dd872fb7 not found: ID does not exist" containerID="2e4ae12584ff08fbdc3511288c31afb36e597f39c42995b993a70211dd872fb7"
Jul 24 16:36:35.925853 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.925821    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e4ae12584ff08fbdc3511288c31afb36e597f39c42995b993a70211dd872fb7"} err="failed to get container status \"2e4ae12584ff08fbdc3511288c31afb36e597f39c42995b993a70211dd872fb7\": rpc error: code = NotFound desc = could not find container \"2e4ae12584ff08fbdc3511288c31afb36e597f39c42995b993a70211dd872fb7\": container with ID starting with 2e4ae12584ff08fbdc3511288c31afb36e597f39c42995b993a70211dd872fb7 not found: ID does not exist"
Jul 24 16:36:35.925853 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.925839    2572 scope.go:117] "RemoveContainer" containerID="e2629b6be0f608bac59a0932ec4dad2da6fa00ecb338eef8632c14150ce50f76"
Jul 24 16:36:35.926095 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:36:35.926080    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2629b6be0f608bac59a0932ec4dad2da6fa00ecb338eef8632c14150ce50f76\": container with ID starting with e2629b6be0f608bac59a0932ec4dad2da6fa00ecb338eef8632c14150ce50f76 not found: ID does not exist" containerID="e2629b6be0f608bac59a0932ec4dad2da6fa00ecb338eef8632c14150ce50f76"
Jul 24 16:36:35.926163 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.926099    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2629b6be0f608bac59a0932ec4dad2da6fa00ecb338eef8632c14150ce50f76"} err="failed to get container status \"e2629b6be0f608bac59a0932ec4dad2da6fa00ecb338eef8632c14150ce50f76\": rpc error: code = NotFound desc = could not find container \"e2629b6be0f608bac59a0932ec4dad2da6fa00ecb338eef8632c14150ce50f76\": container with ID starting with e2629b6be0f608bac59a0932ec4dad2da6fa00ecb338eef8632c14150ce50f76 not found: ID does not exist"
Jul 24 16:36:35.929667 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.929642    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"]
Jul 24 16:36:35.934297 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:35.934273    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-764b09-openshift-default-57cb4b54d4-9qx4b"]
Jul 24 16:36:37.223582 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:37.223551    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55e3284a-cf56-4f90-bb18-d3ee1d754149" path="/var/lib/kubelet/pods/55e3284a-cf56-4f90-bb18-d3ee1d754149/volumes"
Jul 24 16:36:52.227847 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:52.227797    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"]
Jul 24 16:36:52.228341 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:52.228087    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc" podUID="d9fc5ea4-4a14-41fd-addf-51b362924cd3" containerName="istio-proxy" containerID="cri-o://e2b13224f04d9ec765fa6918c32a1654e1f20b6daf28af05e555285d13b79b3f" gracePeriod=30
Jul 24 16:36:57.477547 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.477522    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:36:57.606841 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.606747    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-credential-socket\") pod \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") "
Jul 24 16:36:57.606841 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.606805    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-envoy\") pod \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") "
Jul 24 16:36:57.607100 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.606858    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-podinfo\") pod \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") "
Jul 24 16:36:57.607100 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.606891    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istiod-ca-cert\") pod \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") "
Jul 24 16:36:57.607100 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.606948    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpjf6\" (UniqueName: \"kubernetes.io/projected/d9fc5ea4-4a14-41fd-addf-51b362924cd3-kube-api-access-bpjf6\") pod \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") "
Jul 24 16:36:57.607100 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.606973    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-workload-certs\") pod \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") "
Jul 24 16:36:57.607320 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.607099    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-data\") pod \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") "
Jul 24 16:36:57.607320 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.607107    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d9fc5ea4-4a14-41fd-addf-51b362924cd3" (UID: "d9fc5ea4-4a14-41fd-addf-51b362924cd3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:57.607320 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.607148    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-workload-socket\") pod \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") "
Jul 24 16:36:57.607320 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.607173    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-token\") pod \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\" (UID: \"d9fc5ea4-4a14-41fd-addf-51b362924cd3\") "
Jul 24 16:36:57.607320 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.607245    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d9fc5ea4-4a14-41fd-addf-51b362924cd3" (UID: "d9fc5ea4-4a14-41fd-addf-51b362924cd3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:57.607595 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.607422    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d9fc5ea4-4a14-41fd-addf-51b362924cd3" (UID: "d9fc5ea4-4a14-41fd-addf-51b362924cd3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 24 16:36:57.607595 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.607446    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d9fc5ea4-4a14-41fd-addf-51b362924cd3" (UID: "d9fc5ea4-4a14-41fd-addf-51b362924cd3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:57.607595 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.607488    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istiod-ca-cert\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:57.607595 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.607511    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-workload-certs\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:57.607595 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.607525    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-credential-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:57.608160 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.608132    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-data" (OuterVolumeSpecName: "istio-data") pod "d9fc5ea4-4a14-41fd-addf-51b362924cd3" (UID: "d9fc5ea4-4a14-41fd-addf-51b362924cd3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:57.609337 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.609304    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d9fc5ea4-4a14-41fd-addf-51b362924cd3" (UID: "d9fc5ea4-4a14-41fd-addf-51b362924cd3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 24 16:36:57.609443 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.609353    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d9fc5ea4-4a14-41fd-addf-51b362924cd3" (UID: "d9fc5ea4-4a14-41fd-addf-51b362924cd3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 24 16:36:57.609443 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.609383    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9fc5ea4-4a14-41fd-addf-51b362924cd3-kube-api-access-bpjf6" (OuterVolumeSpecName: "kube-api-access-bpjf6") pod "d9fc5ea4-4a14-41fd-addf-51b362924cd3" (UID: "d9fc5ea4-4a14-41fd-addf-51b362924cd3"). InnerVolumeSpecName "kube-api-access-bpjf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:36:57.609522 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.609459    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-token" (OuterVolumeSpecName: "istio-token") pod "d9fc5ea4-4a14-41fd-addf-51b362924cd3" (UID: "d9fc5ea4-4a14-41fd-addf-51b362924cd3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 24 16:36:57.708405 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.708371    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-podinfo\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:57.708405 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.708398    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpjf6\" (UniqueName: \"kubernetes.io/projected/d9fc5ea4-4a14-41fd-addf-51b362924cd3-kube-api-access-bpjf6\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:57.708405 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.708409    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-data\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:57.708405 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.708417    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-workload-socket\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:57.708686 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.708425    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-token\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:57.708686 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:57.708434    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9fc5ea4-4a14-41fd-addf-51b362924cd3-istio-envoy\") on node \"ip-10-0-133-124.ec2.internal\" DevicePath \"\""
Jul 24 16:36:58.004042 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:58.004011    2572 generic.go:356] "Generic (PLEG): container finished" podID="d9fc5ea4-4a14-41fd-addf-51b362924cd3" containerID="e2b13224f04d9ec765fa6918c32a1654e1f20b6daf28af05e555285d13b79b3f" exitCode=0
Jul 24 16:36:58.004232 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:58.004048    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc" event={"ID":"d9fc5ea4-4a14-41fd-addf-51b362924cd3","Type":"ContainerDied","Data":"e2b13224f04d9ec765fa6918c32a1654e1f20b6daf28af05e555285d13b79b3f"}
Jul 24 16:36:58.004232 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:58.004060    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"
Jul 24 16:36:58.004232 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:58.004090    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc" event={"ID":"d9fc5ea4-4a14-41fd-addf-51b362924cd3","Type":"ContainerDied","Data":"c9830b026b5d7b0082ac4fd4f33bf9ae81e30290924c5c26f33d8a87ec2e6295"}
Jul 24 16:36:58.004232 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:58.004115    2572 scope.go:117] "RemoveContainer" containerID="e2b13224f04d9ec765fa6918c32a1654e1f20b6daf28af05e555285d13b79b3f"
Jul 24 16:36:58.013834 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:58.013813    2572 scope.go:117] "RemoveContainer" containerID="463d7073db1d08205e1ef6fce572e1ff6c5bbad83eee6a21e2c039f9cc8a78a7"
Jul 24 16:36:58.022154 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:58.022135    2572 scope.go:117] "RemoveContainer" containerID="e2b13224f04d9ec765fa6918c32a1654e1f20b6daf28af05e555285d13b79b3f"
Jul 24 16:36:58.022432 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:36:58.022413    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2b13224f04d9ec765fa6918c32a1654e1f20b6daf28af05e555285d13b79b3f\": container with ID starting with e2b13224f04d9ec765fa6918c32a1654e1f20b6daf28af05e555285d13b79b3f not found: ID does not exist" containerID="e2b13224f04d9ec765fa6918c32a1654e1f20b6daf28af05e555285d13b79b3f"
Jul 24 16:36:58.022484 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:58.022442    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2b13224f04d9ec765fa6918c32a1654e1f20b6daf28af05e555285d13b79b3f"} err="failed to get container status \"e2b13224f04d9ec765fa6918c32a1654e1f20b6daf28af05e555285d13b79b3f\": rpc error: code = NotFound desc = could not find container \"e2b13224f04d9ec765fa6918c32a1654e1f20b6daf28af05e555285d13b79b3f\": container with ID starting with e2b13224f04d9ec765fa6918c32a1654e1f20b6daf28af05e555285d13b79b3f not found: ID does not exist"
Jul 24 16:36:58.022484 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:58.022461    2572 scope.go:117] "RemoveContainer" containerID="463d7073db1d08205e1ef6fce572e1ff6c5bbad83eee6a21e2c039f9cc8a78a7"
Jul 24 16:36:58.022680 ip-10-0-133-124 kubenswrapper[2572]: E0724 16:36:58.022662    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"463d7073db1d08205e1ef6fce572e1ff6c5bbad83eee6a21e2c039f9cc8a78a7\": container with ID starting with 463d7073db1d08205e1ef6fce572e1ff6c5bbad83eee6a21e2c039f9cc8a78a7 not found: ID does not exist" containerID="463d7073db1d08205e1ef6fce572e1ff6c5bbad83eee6a21e2c039f9cc8a78a7"
Jul 24 16:36:58.022739 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:58.022690    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"463d7073db1d08205e1ef6fce572e1ff6c5bbad83eee6a21e2c039f9cc8a78a7"} err="failed to get container status \"463d7073db1d08205e1ef6fce572e1ff6c5bbad83eee6a21e2c039f9cc8a78a7\": rpc error: code = NotFound desc = could not find container \"463d7073db1d08205e1ef6fce572e1ff6c5bbad83eee6a21e2c039f9cc8a78a7\": container with ID starting with 463d7073db1d08205e1ef6fce572e1ff6c5bbad83eee6a21e2c039f9cc8a78a7 not found: ID does not exist"
Jul 24 16:36:58.032288 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:58.026421    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"]
Jul 24 16:36:58.032716 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:58.032686    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-82adcb-openshift-default-6b88b56c4d-sxdxc"]
Jul 24 16:36:58.910127 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:58.910043    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-m5lp8_c4f5c8cc-3e31-44e5-aff9-ede71e53f010/server/0.log"
Jul 24 16:36:59.222991 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:59.222893    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9fc5ea4-4a14-41fd-addf-51b362924cd3" path="/var/lib/kubelet/pods/d9fc5ea4-4a14-41fd-addf-51b362924cd3/volumes"
Jul 24 16:36:59.287640 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:59.287605    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d995667b8-dmnvs_dcf7d71a-81b7-46e6-b7da-b9c04935dab7/copy-manifests/0.log"
Jul 24 16:36:59.294522 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:59.294502    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d995667b8-dmnvs_dcf7d71a-81b7-46e6-b7da-b9c04935dab7/manager/0.log"
Jul 24 16:36:59.921583 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:36:59.921560    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748480-pfw2x_7ae7a9e6-71cd-41b8-ab6a-cf2f3248e607/cleanup/0.log"
Jul 24 16:37:02.926071 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:02.926038    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25_fcd3b574-df2d-4ba5-a676-9b17478edede/istio-proxy/0.log"
Jul 24 16:37:03.178711 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:03.178627    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-767449f4cf-4vxsn_5bc3e15d-17f4-4dd9-b13f-93730632f09b/kube-auth-proxy/0.log"
Jul 24 16:37:03.429871 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:03.429775    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mp2m6_b6648611-1f32-4df0-8450-5d3283b76622/istio-proxy/1.log"
Jul 24 16:37:04.553822 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:04.553790    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fwxrrj_43454aa6-3fdf-4d75-b335-3f89d28271f6/main/0.log"
Jul 24 16:37:04.805931 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:04.805833    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-x7lls_8cd82046-ac5f-46f2-817f-985998cc08f4/main/0.log"
Jul 24 16:37:10.623542 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.623503    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f8c76/must-gather-6z8qb"]
Jul 24 16:37:10.623988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.623849    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdf7137c-282c-4c64-84e7-277e09187336" containerName="istio-proxy"
Jul 24 16:37:10.623988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.623862    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdf7137c-282c-4c64-84e7-277e09187336" containerName="istio-proxy"
Jul 24 16:37:10.623988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.623872    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" containerName="istio-proxy"
Jul 24 16:37:10.623988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.623878    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" containerName="istio-proxy"
Jul 24 16:37:10.623988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.623890    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55e3284a-cf56-4f90-bb18-d3ee1d754149" containerName="istio-proxy"
Jul 24 16:37:10.623988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.623895    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="55e3284a-cf56-4f90-bb18-d3ee1d754149" containerName="istio-proxy"
Jul 24 16:37:10.623988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.623901    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55e3284a-cf56-4f90-bb18-d3ee1d754149" containerName="istio-proxy"
Jul 24 16:37:10.623988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.623906    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="55e3284a-cf56-4f90-bb18-d3ee1d754149" containerName="istio-proxy"
Jul 24 16:37:10.623988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.623913    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73ea8d88-5704-42d7-a42f-19504b9c2f32" containerName="revoke-keys"
Jul 24 16:37:10.623988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.623932    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="73ea8d88-5704-42d7-a42f-19504b9c2f32" containerName="revoke-keys"
Jul 24 16:37:10.623988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.623949    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9fc5ea4-4a14-41fd-addf-51b362924cd3" containerName="istio-proxy"
Jul 24 16:37:10.623988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.623954    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9fc5ea4-4a14-41fd-addf-51b362924cd3" containerName="istio-proxy"
Jul 24 16:37:10.623988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.623960    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9fc5ea4-4a14-41fd-addf-51b362924cd3" containerName="istio-proxy"
Jul 24 16:37:10.623988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.623966    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9fc5ea4-4a14-41fd-addf-51b362924cd3" containerName="istio-proxy"
Jul 24 16:37:10.624418 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.624015    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" containerName="istio-proxy"
Jul 24 16:37:10.624418 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.624023    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="73ea8d88-5704-42d7-a42f-19504b9c2f32" containerName="revoke-keys"
Jul 24 16:37:10.624418 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.624030    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9fc5ea4-4a14-41fd-addf-51b362924cd3" containerName="istio-proxy"
Jul 24 16:37:10.624418 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.624036    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="55e3284a-cf56-4f90-bb18-d3ee1d754149" containerName="istio-proxy"
Jul 24 16:37:10.624418 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.624042    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" containerName="istio-proxy"
Jul 24 16:37:10.624418 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.624047    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9fc5ea4-4a14-41fd-addf-51b362924cd3" containerName="istio-proxy"
Jul 24 16:37:10.624418 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.624052    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="55e3284a-cf56-4f90-bb18-d3ee1d754149" containerName="istio-proxy"
Jul 24 16:37:10.624418 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.624109    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" containerName="istio-proxy"
Jul 24 16:37:10.624418 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.624114    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cb5ab5d-6d19-436a-8119-f5d2bce30bb4" containerName="istio-proxy"
Jul 24 16:37:10.627166 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.627144    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f8c76/must-gather-6z8qb"
Jul 24 16:37:10.629824 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.629803    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f8c76\"/\"openshift-service-ca.crt\""
Jul 24 16:37:10.629985 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.629805    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-f8c76\"/\"kube-root-ca.crt\""
Jul 24 16:37:10.629985 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.629846    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-f8c76\"/\"default-dockercfg-6svfq\""
Jul 24 16:37:10.636119 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.636093    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f8c76/must-gather-6z8qb"]
Jul 24 16:37:10.721545 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.721505    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/bad24411-8124-4f35-915a-3597b33c09c1-must-gather-output\") pod \"must-gather-6z8qb\" (UID: \"bad24411-8124-4f35-915a-3597b33c09c1\") " pod="openshift-must-gather-f8c76/must-gather-6z8qb"
Jul 24 16:37:10.721756 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.721560    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksfsd\" (UniqueName: \"kubernetes.io/projected/bad24411-8124-4f35-915a-3597b33c09c1-kube-api-access-ksfsd\") pod \"must-gather-6z8qb\" (UID: \"bad24411-8124-4f35-915a-3597b33c09c1\") " pod="openshift-must-gather-f8c76/must-gather-6z8qb"
Jul 24 16:37:10.822545 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.822505    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksfsd\" (UniqueName: \"kubernetes.io/projected/bad24411-8124-4f35-915a-3597b33c09c1-kube-api-access-ksfsd\") pod \"must-gather-6z8qb\" (UID: \"bad24411-8124-4f35-915a-3597b33c09c1\") " pod="openshift-must-gather-f8c76/must-gather-6z8qb"
Jul 24 16:37:10.822774 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.822597    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/bad24411-8124-4f35-915a-3597b33c09c1-must-gather-output\") pod \"must-gather-6z8qb\" (UID: \"bad24411-8124-4f35-915a-3597b33c09c1\") " pod="openshift-must-gather-f8c76/must-gather-6z8qb"
Jul 24 16:37:10.822962 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.822906    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/bad24411-8124-4f35-915a-3597b33c09c1-must-gather-output\") pod \"must-gather-6z8qb\" (UID: \"bad24411-8124-4f35-915a-3597b33c09c1\") " pod="openshift-must-gather-f8c76/must-gather-6z8qb"
Jul 24 16:37:10.830722 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.830703    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksfsd\" (UniqueName: \"kubernetes.io/projected/bad24411-8124-4f35-915a-3597b33c09c1-kube-api-access-ksfsd\") pod \"must-gather-6z8qb\" (UID: \"bad24411-8124-4f35-915a-3597b33c09c1\") " pod="openshift-must-gather-f8c76/must-gather-6z8qb"
Jul 24 16:37:10.938063 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:10.938023    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f8c76/must-gather-6z8qb"
Jul 24 16:37:11.077018 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:11.076992    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f8c76/must-gather-6z8qb"]
Jul 24 16:37:11.078360 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:37:11.078333    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbad24411_8124_4f35_915a_3597b33c09c1.slice/crio-4a0648a11132de08664ea48076c16034d3c1299568b505c5843ee8504eee258f WatchSource:0}: Error finding container 4a0648a11132de08664ea48076c16034d3c1299568b505c5843ee8504eee258f: Status 404 returned error can't find the container with id 4a0648a11132de08664ea48076c16034d3c1299568b505c5843ee8504eee258f
Jul 24 16:37:12.071579 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:12.071547    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f8c76/must-gather-6z8qb" event={"ID":"bad24411-8124-4f35-915a-3597b33c09c1","Type":"ContainerStarted","Data":"41550e9e5d9652860d5ee45b53e0a327f9787e04722b3cabcfa97764902ba6f9"}
Jul 24 16:37:12.071961 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:12.071588    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f8c76/must-gather-6z8qb" event={"ID":"bad24411-8124-4f35-915a-3597b33c09c1","Type":"ContainerStarted","Data":"4a0648a11132de08664ea48076c16034d3c1299568b505c5843ee8504eee258f"}
Jul 24 16:37:13.079018 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:13.078984    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f8c76/must-gather-6z8qb" event={"ID":"bad24411-8124-4f35-915a-3597b33c09c1","Type":"ContainerStarted","Data":"8090d7b12e3802241889f7d032169ad8722651df1c6d9425aaba4669308ada2f"}
Jul 24 16:37:13.096655 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:13.096591    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-f8c76/must-gather-6z8qb" podStartSLOduration=2.260990349 podStartE2EDuration="3.096571883s" totalImagesPullingTime="835.581534ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 16:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:37:13.096037672 +0000 UTC m=+3112.380365280" watchObservedRunningTime="2026-07-24 16:37:13.096571883 +0000 UTC m=+3112.380899491"
Jul 24 16:37:13.522492 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:13.522457    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-k49df_79ccf6fc-b5a8-4aff-bcb2-c14898719ac4/global-pull-secret-syncer/0.log"
Jul 24 16:37:13.718790 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:13.718751    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-w6mr8_50c1abb8-1abe-4f3d-8dea-7dd210b396ee/konnectivity-agent/0.log"
Jul 24 16:37:13.743276 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:13.743242    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-124.ec2.internal_d00c243b91d89928c6de8f1bbae37e8e/haproxy/0.log"
Jul 24 16:37:21.507186 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:21.507148    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-pkj5s_3f086dcc-d843-4f13-b0e2-b42c11329867/kube-state-metrics/0.log"
Jul 24 16:37:21.523207 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:21.523174    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-pkj5s_3f086dcc-d843-4f13-b0e2-b42c11329867/kube-rbac-proxy-main/0.log"
Jul 24 16:37:21.546004 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:21.545724    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-pkj5s_3f086dcc-d843-4f13-b0e2-b42c11329867/kube-rbac-proxy-self/0.log"
Jul 24 16:37:21.637724 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:21.637690    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bcp9q_3e1253f5-c2bc-4076-b9d9-c2ac082bfa45/node-exporter/0.log"
Jul 24 16:37:21.657443 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:21.657410    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bcp9q_3e1253f5-c2bc-4076-b9d9-c2ac082bfa45/kube-rbac-proxy/0.log"
Jul 24 16:37:21.677141 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:21.677115    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bcp9q_3e1253f5-c2bc-4076-b9d9-c2ac082bfa45/init-textfile/0.log"
Jul 24 16:37:23.805430 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:23.805377    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"]
Jul 24 16:37:23.810995 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:23.810962    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:23.818061 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:23.818023    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"]
Jul 24 16:37:23.852481 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:23.852435    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c042d41b-a413-4571-879a-65eb34a2182c-podres\") pod \"perf-node-gather-daemonset-q4gqp\" (UID: \"c042d41b-a413-4571-879a-65eb34a2182c\") " pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:23.852831 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:23.852808    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c042d41b-a413-4571-879a-65eb34a2182c-proc\") pod \"perf-node-gather-daemonset-q4gqp\" (UID: \"c042d41b-a413-4571-879a-65eb34a2182c\") " pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:23.853055 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:23.853040    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c042d41b-a413-4571-879a-65eb34a2182c-sys\") pod \"perf-node-gather-daemonset-q4gqp\" (UID: \"c042d41b-a413-4571-879a-65eb34a2182c\") " pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:23.853208 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:23.853195    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c042d41b-a413-4571-879a-65eb34a2182c-lib-modules\") pod \"perf-node-gather-daemonset-q4gqp\" (UID: \"c042d41b-a413-4571-879a-65eb34a2182c\") " pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:23.853349 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:23.853334    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzxnc\" (UniqueName: \"kubernetes.io/projected/c042d41b-a413-4571-879a-65eb34a2182c-kube-api-access-tzxnc\") pod \"perf-node-gather-daemonset-q4gqp\" (UID: \"c042d41b-a413-4571-879a-65eb34a2182c\") " pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:23.954565 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:23.954518    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c042d41b-a413-4571-879a-65eb34a2182c-podres\") pod \"perf-node-gather-daemonset-q4gqp\" (UID: \"c042d41b-a413-4571-879a-65eb34a2182c\") " pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:23.954565 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:23.954567    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c042d41b-a413-4571-879a-65eb34a2182c-proc\") pod \"perf-node-gather-daemonset-q4gqp\" (UID: \"c042d41b-a413-4571-879a-65eb34a2182c\") " pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:23.954822 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:23.954611    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c042d41b-a413-4571-879a-65eb34a2182c-sys\") pod \"perf-node-gather-daemonset-q4gqp\" (UID: \"c042d41b-a413-4571-879a-65eb34a2182c\") " pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:23.954822 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:23.954639    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c042d41b-a413-4571-879a-65eb34a2182c-lib-modules\") pod \"perf-node-gather-daemonset-q4gqp\" (UID: \"c042d41b-a413-4571-879a-65eb34a2182c\") " pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:23.954822 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:23.954661    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzxnc\" (UniqueName: \"kubernetes.io/projected/c042d41b-a413-4571-879a-65eb34a2182c-kube-api-access-tzxnc\") pod \"perf-node-gather-daemonset-q4gqp\" (UID: \"c042d41b-a413-4571-879a-65eb34a2182c\") " pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:23.954822 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:23.954729    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c042d41b-a413-4571-879a-65eb34a2182c-sys\") pod \"perf-node-gather-daemonset-q4gqp\" (UID: \"c042d41b-a413-4571-879a-65eb34a2182c\") " pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:23.954822 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:23.954754    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c042d41b-a413-4571-879a-65eb34a2182c-proc\") pod \"perf-node-gather-daemonset-q4gqp\" (UID: \"c042d41b-a413-4571-879a-65eb34a2182c\") " pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:23.955071 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:23.954838    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c042d41b-a413-4571-879a-65eb34a2182c-lib-modules\") pod \"perf-node-gather-daemonset-q4gqp\" (UID: \"c042d41b-a413-4571-879a-65eb34a2182c\") " pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:23.955071 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:23.954898    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c042d41b-a413-4571-879a-65eb34a2182c-podres\") pod \"perf-node-gather-daemonset-q4gqp\" (UID: \"c042d41b-a413-4571-879a-65eb34a2182c\") " pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:23.968988 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:23.968944    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzxnc\" (UniqueName: \"kubernetes.io/projected/c042d41b-a413-4571-879a-65eb34a2182c-kube-api-access-tzxnc\") pod \"perf-node-gather-daemonset-q4gqp\" (UID: \"c042d41b-a413-4571-879a-65eb34a2182c\") " pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:24.131657 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:24.131547    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:24.324420 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:24.324387    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"]
Jul 24 16:37:24.327113 ip-10-0-133-124 kubenswrapper[2572]: W0724 16:37:24.327046    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc042d41b_a413_4571_879a_65eb34a2182c.slice/crio-7a7b520d1544fc3e19bfa50fc01c7c2832ce155ce51681a198eada752dd5851c WatchSource:0}: Error finding container 7a7b520d1544fc3e19bfa50fc01c7c2832ce155ce51681a198eada752dd5851c: Status 404 returned error can't find the container with id 7a7b520d1544fc3e19bfa50fc01c7c2832ce155ce51681a198eada752dd5851c
Jul 24 16:37:25.164133 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:25.163835    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp" event={"ID":"c042d41b-a413-4571-879a-65eb34a2182c","Type":"ContainerStarted","Data":"5c9f9a730b11eccfd979a69ad0083b7c6c8f33a93e85a90303961fbc1fde0092"}
Jul 24 16:37:25.164133 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:25.163908    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp" event={"ID":"c042d41b-a413-4571-879a-65eb34a2182c","Type":"ContainerStarted","Data":"7a7b520d1544fc3e19bfa50fc01c7c2832ce155ce51681a198eada752dd5851c"}
Jul 24 16:37:25.164133 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:25.164118    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:25.182017 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:25.181956    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp" podStartSLOduration=2.181933746 podStartE2EDuration="2.181933746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 16:37:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 16:37:25.179550332 +0000 UTC m=+3124.463877952" watchObservedRunningTime="2026-07-24 16:37:25.181933746 +0000 UTC m=+3124.466261355"
Jul 24 16:37:26.332509 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:26.332468    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8wnvz_5746ff45-485c-498a-904e-8c506e3cffc4/dns/0.log"
Jul 24 16:37:26.351735 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:26.351679    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8wnvz_5746ff45-485c-498a-904e-8c506e3cffc4/kube-rbac-proxy/0.log"
Jul 24 16:37:26.499495 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:26.499464    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5xwjc_f6f39b6e-359d-4a0a-9b1e-4b1afa097802/dns-node-resolver/0.log"
Jul 24 16:37:27.071070 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:27.071036    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5f48ddd4fd-n7qdz_0bcfb694-899b-4e73-b76e-69e21015bd67/registry/0.log"
Jul 24 16:37:27.093991 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:27.093959    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4lpwk_ef6c9d06-124e-4ca0-8bb4-dacf53bb2b94/node-ca/0.log"
Jul 24 16:37:28.058301 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:28.058267    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25_fcd3b574-df2d-4ba5-a676-9b17478edede/istio-proxy/0.log"
Jul 24 16:37:28.344623 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:28.344534    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-767449f4cf-4vxsn_5bc3e15d-17f4-4dd9-b13f-93730632f09b/kube-auth-proxy/0.log"
Jul 24 16:37:28.402644 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:28.402613    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mp2m6_b6648611-1f32-4df0-8450-5d3283b76622/istio-proxy/0.log"
Jul 24 16:37:28.453553 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:28.453517    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mp2m6_b6648611-1f32-4df0-8450-5d3283b76622/istio-proxy/1.log"
Jul 24 16:37:29.345755 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:29.345678    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5jtpt_acb09411-3411-4d23-875c-2b4768a9a968/serve-healthcheck-canary/0.log"
Jul 24 16:37:30.065803 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:30.065763    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z79q4_ba905f82-a2c3-4b61-b253-d7a66300d7c2/kube-rbac-proxy/0.log"
Jul 24 16:37:30.086481 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:30.086453    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z79q4_ba905f82-a2c3-4b61-b253-d7a66300d7c2/exporter/0.log"
Jul 24 16:37:30.107954 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:30.107903    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-z79q4_ba905f82-a2c3-4b61-b253-d7a66300d7c2/extractor/0.log"
Jul 24 16:37:31.181298 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:31.181258    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-f8c76/perf-node-gather-daemonset-q4gqp"
Jul 24 16:37:33.079131 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:33.079098    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-m5lp8_c4f5c8cc-3e31-44e5-aff9-ede71e53f010/server/0.log"
Jul 24 16:37:33.321568 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:33.321533    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d995667b8-dmnvs_dcf7d71a-81b7-46e6-b7da-b9c04935dab7/manager/0.log"
Jul 24 16:37:33.347242 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:33.347217    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d995667b8-dmnvs_dcf7d71a-81b7-46e6-b7da-b9c04935dab7/copy-manifests/0.log"
Jul 24 16:37:34.243286 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:34.243254    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25_fcd3b574-df2d-4ba5-a676-9b17478edede/istio-proxy/0.log"
Jul 24 16:37:34.535222 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:34.535193    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-767449f4cf-4vxsn_5bc3e15d-17f4-4dd9-b13f-93730632f09b/kube-auth-proxy/0.log"
Jul 24 16:37:34.571632 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:34.571528    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mp2m6_b6648611-1f32-4df0-8450-5d3283b76622/istio-proxy/0.log"
Jul 24 16:37:34.606843 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:34.606811    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mp2m6_b6648611-1f32-4df0-8450-5d3283b76622/istio-proxy/1.log"
Jul 24 16:37:34.913099 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:34.913062    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-57bd9ccdc-rmz4b_8b3fe002-5579-4051-8c41-589b1471cab7/manager/0.log"
Jul 24 16:37:36.156119 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:36.156088    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-8ft25_fcd3b574-df2d-4ba5-a676-9b17478edede/istio-proxy/0.log"
Jul 24 16:37:36.447617 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:36.447587    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-767449f4cf-4vxsn_5bc3e15d-17f4-4dd9-b13f-93730632f09b/kube-auth-proxy/0.log"
Jul 24 16:37:36.486386 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:36.486358    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mp2m6_b6648611-1f32-4df0-8450-5d3283b76622/istio-proxy/0.log"
Jul 24 16:37:36.543900 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:36.543867    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-mp2m6_b6648611-1f32-4df0-8450-5d3283b76622/istio-proxy/1.log"
Jul 24 16:37:40.030224 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:40.030181    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-v8c2j_568f5f8a-4f51-4104-9d52-f9495186f9ad/migrator/0.log"
Jul 24 16:37:40.049021 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:40.048990    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-v8c2j_568f5f8a-4f51-4104-9d52-f9495186f9ad/graceful-termination/0.log"
Jul 24 16:37:41.461405 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:41.461377    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ck8qc_46f61359-1f52-4ef6-9831-b55a3e3096e5/kube-multus-additional-cni-plugins/0.log"
Jul 24 16:37:41.481322 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:41.481290    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ck8qc_46f61359-1f52-4ef6-9831-b55a3e3096e5/egress-router-binary-copy/0.log"
Jul 24 16:37:41.504079 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:41.504051    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ck8qc_46f61359-1f52-4ef6-9831-b55a3e3096e5/cni-plugins/0.log"
Jul 24 16:37:41.528887 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:41.528856    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ck8qc_46f61359-1f52-4ef6-9831-b55a3e3096e5/bond-cni-plugin/0.log"
Jul 24 16:37:41.549147 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:41.549118    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ck8qc_46f61359-1f52-4ef6-9831-b55a3e3096e5/routeoverride-cni/0.log"
Jul 24 16:37:41.576439 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:41.576406    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ck8qc_46f61359-1f52-4ef6-9831-b55a3e3096e5/whereabouts-cni-bincopy/0.log"
Jul 24 16:37:41.596901 ip-10-0-133-124 kubenswrapper[2572]: I0724 16:37:41.596873    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ck8qc_46f61359-1f52-4ef6-9831-b55a3e3096e5/whereabouts-cni/0.log"