Jul 11 23:31:08.949427 ip-10-0-136-83 systemd[1]: Starting Kubernetes Kubelet...
Jul 11 23:31:09.425560 ip-10-0-136-83 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 11 23:31:09.425560 ip-10-0-136-83 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 11 23:31:09.425560 ip-10-0-136-83 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 11 23:31:09.425560 ip-10-0-136-83 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 11 23:31:09.425560 ip-10-0-136-83 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 11 23:31:09.425560 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.400277    2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 11 23:31:09.425560 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406442    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 11 23:31:09.425560 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406457    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 11 23:31:09.425560 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406461    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 11 23:31:09.425560 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406465    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 11 23:31:09.425560 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406468    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 11 23:31:09.425560 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406471    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 11 23:31:09.425560 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406474    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 11 23:31:09.425560 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406477    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406480    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406483    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406486    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406488    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406491    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406494    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406496    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406500    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406504    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406506    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406509    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406513    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406516    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406519    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406522    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406526    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406530    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406533    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 11 23:31:09.426852 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406536    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406539    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406542    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406545    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406548    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406551    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406553    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406556    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406559    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406561    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406564    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406567    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406569    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406572    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406574    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406577    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406580    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406582    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406585    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406587    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 11 23:31:09.427931 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406590    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406592    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406595    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406598    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406601    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406604    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406606    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406609    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406611    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406614    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406617    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406620    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406623    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406625    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406628    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406630    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406633    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406635    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406639    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406642    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 11 23:31:09.428788 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406644    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406647    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406650    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406653    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406655    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406660    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406664    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406667    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406670    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406673    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406675    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406678    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406680    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406683    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406686    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406688    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406691    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406694    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406696    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406699    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 11 23:31:09.429881 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406702    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406706    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406709    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406712    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406714    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406717    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406720    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406722    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406725    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406727    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406730    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406733    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.406735    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408347    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408356    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408360    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408363    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408366    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408370    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408372    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 11 23:31:09.430689 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408376    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408379    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408382    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408384    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408387    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408390    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408392    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408395    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408397    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408400    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408402    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408405    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408407    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408411    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408415    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408418    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408420    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408423    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408425    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 11 23:31:09.431760 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408428    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408432    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408436    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408439    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408442    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408445    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408448    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408451    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408454    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408456    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408459    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408462    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408468    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408470    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408473    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408476    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408479    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408481    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408484    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408487    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 11 23:31:09.432620 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408489    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408492    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408494    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408496    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408499    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408502    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408504    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408507    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408509    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408512    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408514    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408521    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408524    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408527    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408529    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408532    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408534    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408537    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408539    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408542    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 11 23:31:09.433929 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408545    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408547    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408550    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408553    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408555    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408558    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408561    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408564    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408567    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408571    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408574    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408577    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408580    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408583    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408585    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408587    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408590    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408592    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408595    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 11 23:31:09.434807 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408598    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408600    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408603    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408605    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408609    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408611    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408614    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408617    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408619    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408624    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408626    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408629    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408631    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.408634    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409040    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409045    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409049    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409052    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409058    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409060    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 11 23:31:09.435897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409064    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 11 23:31:09.436759 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409066    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 11 23:31:09.436759 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409069    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 11 23:31:09.436759 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409071    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 11 23:31:09.436759 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409074    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 11 23:31:09.436759 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409077    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 11 23:31:09.436759 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409079    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 11 23:31:09.436759 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409082    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 11 23:31:09.436759 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409085    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 11 23:31:09.436759 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409087    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 11 23:31:09.436759 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409090    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 11 23:31:09.436759 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409092    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 11 23:31:09.436759 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409096    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 11 23:31:09.436759 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409100    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 11 23:31:09.436759 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409103    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 11 23:31:09.436759 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409106    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 11 23:31:09.436759 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409110    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 11 23:31:09.436759 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409114    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 11 23:31:09.436759 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409117    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409120    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409123    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409127    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409130    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409132    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409135    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409137    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409140    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409143    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409145    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409162    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409165    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409176    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409179    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409181    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409184    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409186    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409189    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409193    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 11 23:31:09.437836 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409195    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409198    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409200    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409202    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409205    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409208    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409210    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409212    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409215    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409218    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409222    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409224    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409227    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409229    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409232    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409237    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409240    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409242    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409245    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409247    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 11 23:31:09.438700 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409250    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409252    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409255    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409257    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409260    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409262    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409270    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409273    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409276    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409278    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409281    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409283    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409285    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409288    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409291    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409294    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409296    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409298    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409301    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409303    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 11 23:31:09.439992 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409306    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409309    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409312    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409315    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409317    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409320    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409322    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409326    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409329    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409331    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409334    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409336    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409339    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409341    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.409344    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409409    2569 flags.go:64] FLAG: --address="0.0.0.0"
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409416    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409429    2569 flags.go:64] FLAG: --anonymous-auth="true"
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409434    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409443    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409447    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409451    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 11 23:31:09.440844 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409455    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409458    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409461    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409474    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409478    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409482    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409487    2569 flags.go:64] FLAG: --cgroup-root=""
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409491    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409495    2569 flags.go:64] FLAG: --client-ca-file=""
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409499    2569 flags.go:64] FLAG: --cloud-provider="external"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409504    2569 flags.go:64] FLAG: --cluster-dns="[]"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409512    2569 flags.go:64] FLAG: --cluster-domain=""
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409518    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409522    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409527    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409532    2569 flags.go:64] FLAG: --container-log-max-files="5"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409537    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409544    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409548    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409553    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409558    2569 flags.go:64] FLAG: --contention-profiling="false"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409563    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409567    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409572    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 11 23:31:09.442774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409576    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409582    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409586    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409590    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409594    2569 flags.go:64] FLAG: --enable-load-reader="false"
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409599    2569 flags.go:64] FLAG: --enable-server="true"
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409603    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409610    2569 flags.go:64] FLAG: --event-burst="100"
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409614    2569 flags.go:64] FLAG: --event-qps="50"
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409618    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409623    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409627    2569 flags.go:64] FLAG: --eviction-hard=""
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409632    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409636    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409641    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409645    2569 flags.go:64] FLAG: --eviction-soft=""
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409650    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409654    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409658    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409662    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409668    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409672    2569 flags.go:64] FLAG: --fail-swap-on="true"
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409676    2569 flags.go:64] FLAG: --feature-gates=""
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409682    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409686    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 11 23:31:09.443857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409692    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409696    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409701    2569 flags.go:64] FLAG: --healthz-port="10248"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409705    2569 flags.go:64] FLAG: --help="false"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409710    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-136-83.ec2.internal"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409715    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409720    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409724    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409729    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409735    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409739    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409743    2569 flags.go:64] FLAG: --image-service-endpoint=""
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409746    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409749    2569 flags.go:64] FLAG: --kube-api-burst="100"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409756    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409761    2569 flags.go:64] FLAG: --kube-api-qps="50"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409765    2569 flags.go:64] FLAG: --kube-reserved=""
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409769    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409773    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409778    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409782    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409786    2569 flags.go:64] FLAG: --lock-file=""
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409790    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409795    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 11 23:31:09.445566 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409799    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409807    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409811    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409817    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409821    2569 flags.go:64] FLAG: --logging-format="text"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409825    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409830    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409833    2569 flags.go:64] FLAG: --manifest-url=""
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409840    2569 flags.go:64] FLAG: --manifest-url-header=""
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409851    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409856    2569 flags.go:64] FLAG: --max-open-files="1000000"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409862    2569 flags.go:64] FLAG: --max-pods="110"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409866    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409870    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409874    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409879    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409884    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409888    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409892    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409902    2569 flags.go:64] FLAG: --node-status-max-images="50"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409906    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409910    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409915    2569 flags.go:64] FLAG: --pod-cidr=""
Jul 11 23:31:09.446633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409920    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409928    2569 flags.go:64] FLAG: --pod-manifest-path=""
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409932    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409937    2569 flags.go:64] FLAG: --pods-per-core="0"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409941    2569 flags.go:64] FLAG: --port="10250"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409946    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409950    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ffc11429e8f08596"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409955    2569 flags.go:64] FLAG: --qos-reserved=""
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409958    2569 flags.go:64] FLAG: --read-only-port="10255"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409962    2569 flags.go:64] FLAG: --register-node="true"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409966    2569 flags.go:64] FLAG: --register-with-taints=""
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409972    2569 flags.go:64] FLAG: --registry-burst="10"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409977    2569 flags.go:64] FLAG: --registry-qps="5"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409983    2569 flags.go:64] FLAG: --reserved-cpus=""
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409987    2569 flags.go:64] FLAG: --reserved-memory=""
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409992    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.409996    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410003    2569 flags.go:64] FLAG: --rotate-certificates="false"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410007    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410011    2569 flags.go:64] FLAG: --runonce="false"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410015    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410019    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410023    2569 flags.go:64] FLAG: --seccomp-default="false"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410028    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410032    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410036    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 11 23:31:09.447721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410042    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410046    2569 flags.go:64] FLAG: --storage-driver-password="root"
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410051    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410055    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410059    2569 flags.go:64] FLAG: --storage-driver-user="root"
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410063    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410067    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410073    2569 flags.go:64] FLAG: --system-cgroups=""
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410077    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410086    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410090    2569 flags.go:64] FLAG: --tls-cert-file=""
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410094    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410100    2569 flags.go:64] FLAG: --tls-min-version=""
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410104    2569 flags.go:64] FLAG: --tls-private-key-file=""
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410107    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410112    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410116    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410120    2569 flags.go:64] FLAG: --v="2"
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410125    2569 flags.go:64] FLAG: --version="false"
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410132    2569 flags.go:64] FLAG: --vmodule=""
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410137    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.410141    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410323    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410330    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 11 23:31:09.448820 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410335    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410340    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410344    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410348    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410352    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410357    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410360    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410364    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410368    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410374    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410380    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410385    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410388    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410391    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410393    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410397    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410402    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410406    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410409    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410414    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 11 23:31:09.451190 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410418    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410421    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410425    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410429    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410432    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410437    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410441    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410446    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410450    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410454    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410458    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410463    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410467    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410471    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410475    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410479    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410483    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410487    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410491    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410494    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 11 23:31:09.452447 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410498    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410502    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410508    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410513    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410518    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410522    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410526    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410530    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410534    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410538    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410542    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410545    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410549    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410553    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410556    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410560    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410564    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410567    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410571    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 11 23:31:09.453733 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410576    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410580    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410583    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410587    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410592    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410596    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410600    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410603    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410607    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410611    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410614    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410616    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410620    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410624    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410627    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410631    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410635    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410639    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410642    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410647    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 11 23:31:09.454574 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410650    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 11 23:31:10.982460 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410654    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 11 23:31:10.982460 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410657    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 11 23:31:10.982460 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410662    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 11 23:31:10.982460 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410666    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 11 23:31:10.982460 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410669    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 11 23:31:10.982460 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410673    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 11 23:31:10.982460 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410677    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 11 23:31:10.982460 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410681    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 11 23:31:10.982460 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410684    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 11 23:31:10.982460 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410687    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 11 23:31:10.982460 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410692    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 11 23:31:10.982460 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410697    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 11 23:31:10.982460 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410703    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 11 23:31:10.982460 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410707    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 11 23:31:10.982460 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410711    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 11 23:31:10.982460 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410717    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 11 23:31:10.982460 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.410720    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 11 23:31:09.504952 ip-10-0-136-83 systemd[1]: Started Kubernetes Kubelet.
Jul 11 23:31:11.007145 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.411480    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 11 23:31:11.007145 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.419263    2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 11 23:31:11.007145 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.419283    2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 11 23:31:11.007145 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.420391    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 11 23:31:11.007145 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.420401    2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 11 23:31:11.007145 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420488    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 11 23:31:11.007145 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420496    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 11 23:31:11.007145 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420500    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 11 23:31:11.007145 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420504    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 11 23:31:11.007145 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420509    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 11 23:31:11.007145 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420513    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 11 23:31:11.007145 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420516    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 11 23:31:11.007145 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420520    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 11 23:31:11.007145 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420524    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 11 23:31:11.007145 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420527    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420531    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420535    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420538    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420541    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420545    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420551    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420557    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420562    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420566    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420570    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420574    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420578    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420582    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420586    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420590    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420593    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420598    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420601    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 11 23:31:11.007966 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420605    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420618    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420623    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420627    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420631    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420635    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420638    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420643    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420646    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420650    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420654    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420658    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420662    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420666    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420669    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420673    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420677    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420681    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420685    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420688    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 11 23:31:11.008891 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420693    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420697    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420701    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420705    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420709    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420714    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420717    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420721    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420725    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420729    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420732    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420736    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420740    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420743    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420754    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420758    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420762    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420766    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420769    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420773    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 11 23:31:11.009618 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420777    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420780    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420784    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420788    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420792    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420795    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420799    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420803    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420806    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420810    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420813    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420819    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420824    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420829    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420836    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420840    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420845    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420849    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420853    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 11 23:31:11.010256 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420857    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 11 23:31:11.011011 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420862    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 11 23:31:11.011011 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420866    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 11 23:31:11.011011 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420870    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 11 23:31:11.011011 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420874    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 11 23:31:11.011011 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420878    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 11 23:31:11.011011 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420882    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 11 23:31:11.011011 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420887    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 11 23:31:11.011011 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420900    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 11 23:31:11.011011 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420904    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 11 23:31:11.011011 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420907    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 11 23:31:11.011011 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.420911    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 11 23:31:11.011011 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.420918    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 11 23:31:11.011011 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421115    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 11 23:31:11.011011 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421122    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421127    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421131    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421135    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421138    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421143    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421146    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421163    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421167    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421170    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421174    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421182    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421186    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421190    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421196    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421200    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421204    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421207    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421211    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421214    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 11 23:31:11.011524 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421218    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421221    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421225    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421229    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421232    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421236    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421240    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421251    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421255    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421259    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421263    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421267    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421270    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421274    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421278    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421281    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421285    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421289    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421292    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421296    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 11 23:31:11.012298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421300    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421304    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421309    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421315    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421319    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421322    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421328    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421332    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421335    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421339    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421343    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421346    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421350    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421354    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421358    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421362    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421365    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421369    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421373    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 11 23:31:11.013077 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421376    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421388    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421392    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421395    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421399    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421403    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421406    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421410    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421414    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421417    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421420    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421424    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421428    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421431    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421435    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421439    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421444    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421447    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421451    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421456    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 11 23:31:11.015007 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421460    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 11 23:31:11.016063 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421471    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 11 23:31:11.016063 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421475    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 11 23:31:11.016063 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421479    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 11 23:31:11.016063 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421483    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 11 23:31:11.016063 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421486    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 11 23:31:11.016063 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421491    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 11 23:31:11.016063 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421497    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 11 23:31:11.016063 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421500    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 11 23:31:11.016063 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421504    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 11 23:31:11.016063 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421508    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 11 23:31:11.016063 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421519    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 11 23:31:11.016063 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421523    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 11 23:31:11.016063 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421527    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 11 23:31:11.016063 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421537    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 11 23:31:11.016063 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421540    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 11 23:31:11.016063 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421544    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 11 23:31:11.016063 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421548    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 11 23:31:11.016063 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:09.421552    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 11 23:31:11.017789 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.421559    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 11 23:31:11.017789 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.421703    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 11 23:31:11.017789 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.425314    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 11 23:31:11.017789 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.426810    2569 server.go:1019] "Starting client certificate rotation"
Jul 11 23:31:11.017789 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.429365    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 11 23:31:11.017789 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.429390    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 11 23:31:11.017789 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.433205    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 11 23:31:11.017789 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.455935    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 11 23:31:11.017789 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.470650    2569 log.go:25] "Validated CRI v1 runtime API"
Jul 11 23:31:11.017789 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.472814    2569 log.go:25] "Validated CRI v1 image API"
Jul 11 23:31:11.017789 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.474815    2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 11 23:31:11.017789 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.477397    2569 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ab3cdc74-7ad3-4c9c-8fa8-71266faa77f5:/dev/nvme0n1p4 c39ebf6e-36a4-40a9-af75-6ab23f79e248:/dev/nvme0n1p3]
Jul 11 23:31:11.018226 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.477417    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 11 23:31:11.018226 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.481755    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 11 23:31:11.018690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.486040    2569 manager.go:216] Machine: {Timestamp:2026-07-11 23:31:09.484847347 +0000 UTC m=+0.398214391 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101567 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2875d1a3ddf402a3a82c29880f6d4e SystemUUID:ec2875d1-a3dd-f402-a3a8-2c29880f6d4e BootID:212137de-8d03-4535-8794-9f6b2fbc5ace Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ab:0c:fb:b1:fb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ab:0c:fb:b1:fb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:82:1f:32:02:53:a8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 11 23:31:11.018690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.486143    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 11 23:31:11.018690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.486237    2569 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 11 23:31:11.018690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.487367    2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 11 23:31:11.018690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.487390    2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-83.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 11 23:31:11.018690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.488072    2569 topology_manager.go:138] "Creating topology manager with none policy"
Jul 11 23:31:11.018690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.488082    2569 container_manager_linux.go:309] "Creating device plugin manager"
Jul 11 23:31:11.018690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.488091    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 11 23:31:11.018690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.488848    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 11 23:31:11.018690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.490073    2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 11 23:31:11.018690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.493861    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 11 23:31:11.018690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.493982    2569 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 11 23:31:11.018690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.494007    2569 kubelet.go:486] "Attempting to sync node with API server"
Jul 11 23:31:11.018690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.494017    2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 11 23:31:11.018690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.494031    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 11 23:31:11.018690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.494048    2569 kubelet.go:398] "Adding apiserver pod source"
Jul 11 23:31:11.018690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.494107    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 11 23:31:11.018690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.494633    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.494658    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.496892    2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.498635    2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.498685    2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.499441    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.499461    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.499472    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.499493    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.499506    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.499517    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.499694    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.499702    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.499708    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.499715    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.499730    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.499739    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.500580    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.500590    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.503364    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mfrkh"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.504077    2569 server.go:1268] "Started kubelet"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.504719    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.504778    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.504846    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:09.508753    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-83.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:09.508972    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.509183    2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.509197    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mfrkh"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.509848    2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-83.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:09.511655    2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.511896    2569 server.go:310] "Adding debug handlers to kubelet server"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:09.513313    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-83.ec2.internal.18c1602bffa0fc7a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-83.ec2.internal,UID:ip-10-0-136-83.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-83.ec2.internal,},FirstTimestamp:2026-07-11 23:31:09.50402777 +0000 UTC m=+0.417394813,LastTimestamp:2026-07-11 23:31:09.50402777 +0000 UTC m=+0.417394813,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-83.ec2.internal,}"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.514529    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.514593    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:09.514643    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-83.ec2.internal\" not found"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.515176    2569 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.515191    2569 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.515214    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.515281    2569 reconstruct.go:97] "Volume reconstruction finished"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.515289    2569 reconciler.go:29] "Reconciler: start to sync state"
Jul 11 23:31:11.019398 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.516167    2569 factory.go:55] Registering systemd factory
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.516183    2569 factory.go:223] Registration of the systemd container factory successfully
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.516447    2569 factory.go:153] Registering CRI-O factory
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.516459    2569 factory.go:223] Registration of the crio container factory successfully
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.516498    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.516521    2569 factory.go:103] Registering Raw factory
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.516536    2569 manager.go:1196] Started watching for new ooms in manager
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.517040    2569 manager.go:349] Starting recovery of all containers
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.525165    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:09.527790    2569 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-83.ec2.internal\" not found" node="ip-10-0-136-83.ec2.internal"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.527863    2569 manager.go:354] Recovery completed
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:09.529097    2569 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.532578    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.534456    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasSufficientMemory"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.534486    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasNoDiskPressure"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.534499    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasSufficientPID"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.535030    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.535040    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.535040    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.535123    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.535695    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasSufficientMemory"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.535720    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasNoDiskPressure"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.535734    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasSufficientPID"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.537217    2569 policy_none.go:49] "None policy: Start"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.537230    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:09.537241    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:10.874417    2569 policy_none.go:47] "Start"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:10.912821    2569 manager.go:337] "Starting Device Plugin manager"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:10.912854    2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:10.912863    2569 server.go:87] "Starting device plugin registration server"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:10.913109    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:10.913119    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:10.913216    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:10.913304    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:10.913314    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:10.913825    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:10.913883    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-83.ec2.internal\" not found"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:10.975390    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:10.976501    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:10.976534    2569 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:10.976557    2569 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:10.976589    2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:10.978468    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.014220    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.015446    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasSufficientMemory"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.015479    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasNoDiskPressure"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.015494    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasSufficientPID"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.015526    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-83.ec2.internal"
Jul 11 23:31:11.170208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.022938    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-83.ec2.internal"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:11.022960    2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-83.ec2.internal\": node \"ip-10-0-136-83.ec2.internal\" not found"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:11.039182    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-83.ec2.internal\" not found"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.076926    2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-83.ec2.internal"]
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.077057    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.078740    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasSufficientMemory"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.078777    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasNoDiskPressure"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.078796    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasSufficientPID"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.081217    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.081323    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.081353    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.083960    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasSufficientMemory"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.083972    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasSufficientMemory"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.083984    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasNoDiskPressure"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.083991    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasNoDiskPressure"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.083996    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasSufficientPID"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.084000    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasSufficientPID"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.086229    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-83.ec2.internal"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.086252    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.086920    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasSufficientMemory"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.086954    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasNoDiskPressure"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.086969    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeHasSufficientPID"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:11.101968    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-83.ec2.internal\" not found" node="ip-10-0-136-83.ec2.internal"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:11.105464    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-83.ec2.internal\" not found" node="ip-10-0-136-83.ec2.internal"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:11.139251    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-83.ec2.internal\" not found"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.151859    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42968f545a222e19521d04bf195c5440-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal\" (UID: \"42968f545a222e19521d04bf195c5440\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.151891    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42968f545a222e19521d04bf195c5440-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal\" (UID: \"42968f545a222e19521d04bf195c5440\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal"
Jul 11 23:31:11.171838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.151913    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6c9fb79e71283f9e071707f8e0898820-config\") pod \"kube-apiserver-proxy-ip-10-0-136-83.ec2.internal\" (UID: \"6c9fb79e71283f9e071707f8e0898820\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-83.ec2.internal"
Jul 11 23:31:11.298186 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:11.239318    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-83.ec2.internal\" not found"
Jul 11 23:31:11.298186 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.252681    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42968f545a222e19521d04bf195c5440-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal\" (UID: \"42968f545a222e19521d04bf195c5440\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal"
Jul 11 23:31:11.298186 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.252708    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42968f545a222e19521d04bf195c5440-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal\" (UID: \"42968f545a222e19521d04bf195c5440\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal"
Jul 11 23:31:11.298186 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.252731    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6c9fb79e71283f9e071707f8e0898820-config\") pod \"kube-apiserver-proxy-ip-10-0-136-83.ec2.internal\" (UID: \"6c9fb79e71283f9e071707f8e0898820\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-83.ec2.internal"
Jul 11 23:31:11.298186 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.252783    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6c9fb79e71283f9e071707f8e0898820-config\") pod \"kube-apiserver-proxy-ip-10-0-136-83.ec2.internal\" (UID: \"6c9fb79e71283f9e071707f8e0898820\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-83.ec2.internal"
Jul 11 23:31:11.298186 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.252795    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42968f545a222e19521d04bf195c5440-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal\" (UID: \"42968f545a222e19521d04bf195c5440\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal"
Jul 11 23:31:11.298186 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.252795    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42968f545a222e19521d04bf195c5440-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal\" (UID: \"42968f545a222e19521d04bf195c5440\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal"
Jul 11 23:31:11.488565 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:11.340122    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-83.ec2.internal\" not found"
Jul 11 23:31:11.488565 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.403568    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal"
Jul 11 23:31:11.488565 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.408988    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-83.ec2.internal"
Jul 11 23:31:11.488565 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:11.440727    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-83.ec2.internal\" not found"
Jul 11 23:31:11.657108 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:11.541221    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-83.ec2.internal\" not found"
Jul 11 23:31:11.657108 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:11.641796    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-83.ec2.internal\" not found"
Jul 11 23:31:11.742440 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:11.742390    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-83.ec2.internal\" not found"
Jul 11 23:31:11.761866 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.761842    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 11 23:31:11.762344 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.761972    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 11 23:31:11.762344 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.762025    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 11 23:31:11.843398 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:11.843366    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-83.ec2.internal\" not found"
Jul 11 23:31:11.850164 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.850133    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-10 23:26:10 +0000 UTC" deadline="2028-03-13 20:42:27.509672826 +0000 UTC"
Jul 11 23:31:11.850164 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:11.850161    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14661h11m15.659515027s"
Jul 11 23:31:11.943866 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:11.943833    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-83.ec2.internal\" not found"
Jul 11 23:31:12.039541 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:12.039508    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c9fb79e71283f9e071707f8e0898820.slice/crio-e5af311233f69b9538a090bfa5fc1b96429359d304c8e1ea851022494197da4a WatchSource:0}: Error finding container e5af311233f69b9538a090bfa5fc1b96429359d304c8e1ea851022494197da4a: Status 404 returned error can't find the container with id e5af311233f69b9538a090bfa5fc1b96429359d304c8e1ea851022494197da4a
Jul 11 23:31:12.040149 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:12.040126    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42968f545a222e19521d04bf195c5440.slice/crio-5a00d39318e53a689883605ccaac5894956ba5504f43b3f59366fa7591e4f6e6 WatchSource:0}: Error finding container 5a00d39318e53a689883605ccaac5894956ba5504f43b3f59366fa7591e4f6e6: Status 404 returned error can't find the container with id 5a00d39318e53a689883605ccaac5894956ba5504f43b3f59366fa7591e4f6e6
Jul 11 23:31:12.044003 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:12.043984    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-83.ec2.internal\" not found"
Jul 11 23:31:12.044062 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.044050    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 11 23:31:12.144837 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:12.144798    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-83.ec2.internal\" not found"
Jul 11 23:31:12.245315 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:12.245232    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-83.ec2.internal\" not found"
Jul 11 23:31:12.346112 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:12.346073    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-83.ec2.internal\" not found"
Jul 11 23:31:12.364691 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.364662    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 11 23:31:12.429683 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.429629    2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 11 23:31:12.451160 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.451130    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal"
Jul 11 23:31:12.461157 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.461120    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 11 23:31:12.462575 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.462318    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-83.ec2.internal"
Jul 11 23:31:12.469996 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.469968    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 11 23:31:12.568667 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.568476    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 11 23:31:12.830892 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.830829    2569 apiserver.go:52] "Watching apiserver"
Jul 11 23:31:12.840944 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.840187    2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 11 23:31:12.841574 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.841206    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-chgrx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds","openshift-cluster-node-tuning-operator/tuned-9vwnz","openshift-dns/node-resolver-smg2m","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal","openshift-multus/multus-additional-cni-plugins-jwbv4","openshift-multus/network-metrics-daemon-5qc5j","openshift-network-diagnostics/network-check-target-hrfff","kube-system/konnectivity-agent-wh9hv","kube-system/kube-apiserver-proxy-ip-10-0-136-83.ec2.internal","openshift-image-registry/node-ca-mlkk5","openshift-multus/multus-nrn9c","openshift-network-operator/iptables-alerter-8j6d9","openshift-ovn-kubernetes/ovnkube-node-xd2gc"]
Jul 11 23:31:12.843873 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.843849    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:12.843992 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:12.843937    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5qc5j" podUID="7b557007-8a5a-47eb-a23e-a96ed97fb64c"
Jul 11 23:31:12.845952 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.845930    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mlkk5"
Jul 11 23:31:12.848669 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.848229    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.848669 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.848414    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zs5np\""
Jul 11 23:31:12.848669 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.848520    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 11 23:31:12.848669 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.848466    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 11 23:31:12.848979 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.848958    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 11 23:31:12.850347 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.850303    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 11 23:31:12.850492 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.850445    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-smg2m"
Jul 11 23:31:12.851239 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.851048    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wxdp6\""
Jul 11 23:31:12.851239 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.851075    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 11 23:31:12.851239 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.851167    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 11 23:31:12.853017 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.852897    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5nn7w\""
Jul 11 23:31:12.853450 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.853116    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 11 23:31:12.853618 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.853572    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.854551 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.853580    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 11 23:31:12.856092 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.855493    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:12.856415 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.856397    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 11 23:31:12.856547 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.856505    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 11 23:31:12.857375 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.856984    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 11 23:31:12.857375 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.857095    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 11 23:31:12.857375 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.857233    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 11 23:31:12.857375 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.857299    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 11 23:31:12.857668 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.857588    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6gg67\""
Jul 11 23:31:12.857881 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.857863    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wh9hv"
Jul 11 23:31:12.858448 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.858134    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 11 23:31:12.858448 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.858170    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gt2f5\""
Jul 11 23:31:12.858448 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.858301    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 11 23:31:12.858448 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.858342    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 11 23:31:12.858673 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.858486    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 11 23:31:12.858725 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.858679    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 11 23:31:12.860884 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.860416    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8j6d9"
Jul 11 23:31:12.860884 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.860607    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vhqzf\""
Jul 11 23:31:12.860884 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.860657    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 11 23:31:12.861845 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.861588    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 11 23:31:12.861845 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.861700    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 11 23:31:12.861845 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.861716    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-modprobe-d\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.861845 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.861745    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19c676bb-ab21-4b12-8c3f-9183c5f222f7-tmp\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.861845 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.861772    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-run-netns\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.861845 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.861798    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-run-systemd\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.861845 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.861827    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-run-ovn\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.862212 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.861867    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-node-log\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.862212 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.861895    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.862212 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.861945    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffdhf\" (UniqueName: \"kubernetes.io/projected/9f1aaf88-8ea0-4105-b62b-f784d91165d1-kube-api-access-ffdhf\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.862212 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.861972    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-sysctl-conf\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.862212 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862024    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-var-lib-kubelet\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.862212 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862063    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7-tmp-dir\") pod \"node-resolver-smg2m\" (UID: \"fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7\") " pod="openshift-dns/node-resolver-smg2m"
Jul 11 23:31:12.862212 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862090    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-etc-openvswitch\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.862212 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862132    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.862212 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862173    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-cni-bin\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.862212 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862206    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-cni-netd\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.862714 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862228    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f1aaf88-8ea0-4105-b62b-f784d91165d1-ovnkube-config\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.862714 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862264    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790-serviceca\") pod \"node-ca-mlkk5\" (UID: \"ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790\") " pod="openshift-image-registry/node-ca-mlkk5"
Jul 11 23:31:12.862714 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862285    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-run\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.862714 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862306    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-lib-modules\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.862714 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862327    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-host\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.862714 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862351    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-var-lib-openvswitch\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.862714 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862390    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bql6\" (UniqueName: \"kubernetes.io/projected/7b557007-8a5a-47eb-a23e-a96ed97fb64c-kube-api-access-6bql6\") pod \"network-metrics-daemon-5qc5j\" (UID: \"7b557007-8a5a-47eb-a23e-a96ed97fb64c\") " pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:12.862714 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862439    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-tuned\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.862714 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862467    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-log-socket\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.862714 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862492    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f1aaf88-8ea0-4105-b62b-f784d91165d1-ovn-node-metrics-cert\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.862714 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862529    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f1aaf88-8ea0-4105-b62b-f784d91165d1-ovnkube-script-lib\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.862714 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862552    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-systemd\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.862714 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862574    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nthhq\" (UniqueName: \"kubernetes.io/projected/19c676bb-ab21-4b12-8c3f-9183c5f222f7-kube-api-access-nthhq\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.862714 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862594    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 11 23:31:12.863510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862596    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7-hosts-file\") pod \"node-resolver-smg2m\" (UID: \"fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7\") " pod="openshift-dns/node-resolver-smg2m"
Jul 11 23:31:12.863510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862766    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-kubelet\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.863510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862789    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-systemd-units\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.863510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862825    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-slash\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.863510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862852    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs\") pod \"network-metrics-daemon-5qc5j\" (UID: \"7b557007-8a5a-47eb-a23e-a96ed97fb64c\") " pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:12.863510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862882    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790-host\") pod \"node-ca-mlkk5\" (UID: \"ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790\") " pod="openshift-image-registry/node-ca-mlkk5"
Jul 11 23:31:12.863510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862906    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-sysconfig\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.863510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862927    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-kubernetes\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.863510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862949    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-sys\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.863510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.862971    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-run-openvswitch\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.863510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.863001    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-sysctl-d\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.863510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.863027    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgjjb\" (UniqueName: \"kubernetes.io/projected/fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7-kube-api-access-vgjjb\") pod \"node-resolver-smg2m\" (UID: \"fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7\") " pod="openshift-dns/node-resolver-smg2m"
Jul 11 23:31:12.863510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.863050    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f1aaf88-8ea0-4105-b62b-f784d91165d1-env-overrides\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.863510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.863073    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frkw7\" (UniqueName: \"kubernetes.io/projected/ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790-kube-api-access-frkw7\") pod \"node-ca-mlkk5\" (UID: \"ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790\") " pod="openshift-image-registry/node-ca-mlkk5"
Jul 11 23:31:12.863510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.863446    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 11 23:31:12.863510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.863491    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ddvdx\""
Jul 11 23:31:12.864240 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.863773    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 11 23:31:12.864240 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.864048    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:12.871139 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.867052    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 11 23:31:12.871139 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.867282    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 11 23:31:12.871139 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.867970    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tc9wg\""
Jul 11 23:31:12.871139 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.870483    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:12.871139 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:12.870552    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrfff" podUID="12be71b0-f3cc-4a5c-8109-03e7f70bc99f"
Jul 11 23:31:12.871139 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.870953    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 11 23:31:12.874171 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.874052    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:12.874171 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:12.874121    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chgrx" podUID="faee1e90-99c1-4255-adb0-1d56308dbfd2"
Jul 11 23:31:12.876472 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.876235    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:12.878760 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.878488    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-w2rkq\""
Jul 11 23:31:12.878760 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.878584    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 11 23:31:12.880091 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.880053    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9lt8x"
Jul 11 23:31:12.887269 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.887117    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9lt8x"
Jul 11 23:31:12.952192 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.952166    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 11 23:31:12.963534 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963499    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-run-systemd\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.963678 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963545    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-run-ovn\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.963678 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963570    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-node-log\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.963678 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963596    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffdhf\" (UniqueName: \"kubernetes.io/projected/9f1aaf88-8ea0-4105-b62b-f784d91165d1-kube-api-access-ffdhf\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.963678 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963604    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-run-systemd\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.963678 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963622    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7-tmp-dir\") pod \"node-resolver-smg2m\" (UID: \"fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7\") " pod="openshift-dns/node-resolver-smg2m"
Jul 11 23:31:12.963678 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963647    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-etc-openvswitch\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.963678 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963655    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-run-ovn\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.963678 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963671    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.964064 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963679    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-node-log\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.964064 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963695    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-cni-bin\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.964064 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963741    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-cni-bin\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.964064 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963746    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f1aaf88-8ea0-4105-b62b-f784d91165d1-ovnkube-config\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.964064 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963749    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-etc-openvswitch\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.964064 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963775    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.964064 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963788    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcd9p\" (UniqueName: \"kubernetes.io/projected/3680c2ef-7e40-4f46-b1aa-f3581c4f7626-kube-api-access-fcd9p\") pod \"iptables-alerter-8j6d9\" (UID: \"3680c2ef-7e40-4f46-b1aa-f3581c4f7626\") " pod="openshift-network-operator/iptables-alerter-8j6d9"
Jul 11 23:31:12.964064 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963818    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-host-var-lib-cni-multus\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:12.964064 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963842    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/43743170-d713-4977-a555-d59d7ab61f0c-multus-daemon-config\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:12.964064 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963869    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790-serviceca\") pod \"node-ca-mlkk5\" (UID: \"ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790\") " pod="openshift-image-registry/node-ca-mlkk5"
Jul 11 23:31:12.964064 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963893    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-run\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.964064 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963930    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-lib-modules\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.964064 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963946    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-run\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.964064 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963955    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/89063ca1-d3ed-4f06-a51b-e10bc5035759-device-dir\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:12.964064 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963965    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7-tmp-dir\") pod \"node-resolver-smg2m\" (UID: \"fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7\") " pod="openshift-dns/node-resolver-smg2m"
Jul 11 23:31:12.964064 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.963994    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/89063ca1-d3ed-4f06-a51b-e10bc5035759-sys-fs\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:12.964064 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964027    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-system-cni-dir\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:12.964895 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964045    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-os-release\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:12.964895 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964065    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-lib-modules\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.964895 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964065    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bql6\" (UniqueName: \"kubernetes.io/projected/7b557007-8a5a-47eb-a23e-a96ed97fb64c-kube-api-access-6bql6\") pod \"network-metrics-daemon-5qc5j\" (UID: \"7b557007-8a5a-47eb-a23e-a96ed97fb64c\") " pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:12.964895 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964106    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-tuned\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.964895 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964125    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0835749c-5f54-425f-99de-161c4b66c81a-system-cni-dir\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:12.964895 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964141    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0835749c-5f54-425f-99de-161c4b66c81a-cni-binary-copy\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:12.964895 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964157    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret\") pod \"global-pull-secret-syncer-chgrx\" (UID: \"faee1e90-99c1-4255-adb0-1d56308dbfd2\") " pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:12.964895 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964183    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4ae214fa-346c-48ec-b3f7-67071c699d39-agent-certs\") pod \"konnectivity-agent-wh9hv\" (UID: \"4ae214fa-346c-48ec-b3f7-67071c699d39\") " pod="kube-system/konnectivity-agent-wh9hv"
Jul 11 23:31:12.964895 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964207    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4ae214fa-346c-48ec-b3f7-67071c699d39-konnectivity-ca\") pod \"konnectivity-agent-wh9hv\" (UID: \"4ae214fa-346c-48ec-b3f7-67071c699d39\") " pod="kube-system/konnectivity-agent-wh9hv"
Jul 11 23:31:12.964895 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964227    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-etc-kubernetes\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:12.964895 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964257    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7-hosts-file\") pod \"node-resolver-smg2m\" (UID: \"fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7\") " pod="openshift-dns/node-resolver-smg2m"
Jul 11 23:31:12.964895 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964273    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790-serviceca\") pod \"node-ca-mlkk5\" (UID: \"ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790\") " pod="openshift-image-registry/node-ca-mlkk5"
Jul 11 23:31:12.964895 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964280    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-systemd-units\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.964895 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964337    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-systemd-units\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.964895 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964366    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7-hosts-file\") pod \"node-resolver-smg2m\" (UID: \"fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7\") " pod="openshift-dns/node-resolver-smg2m"
Jul 11 23:31:12.964895 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964390    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/89063ca1-d3ed-4f06-a51b-e10bc5035759-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:12.964895 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964464    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 11 23:31:12.965589 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964473    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5wmp\" (UniqueName: \"kubernetes.io/projected/89063ca1-d3ed-4f06-a51b-e10bc5035759-kube-api-access-b5wmp\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:12.965589 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964499    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-host-run-multus-certs\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:12.965589 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964519    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9488l\" (UniqueName: \"kubernetes.io/projected/43743170-d713-4977-a555-d59d7ab61f0c-kube-api-access-9488l\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:12.965589 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964403    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9f1aaf88-8ea0-4105-b62b-f784d91165d1-ovnkube-config\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.965589 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964542    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0835749c-5f54-425f-99de-161c4b66c81a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:12.965589 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964566    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/89063ca1-d3ed-4f06-a51b-e10bc5035759-registration-dir\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:12.965589 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964617    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/faee1e90-99c1-4255-adb0-1d56308dbfd2-dbus\") pod \"global-pull-secret-syncer-chgrx\" (UID: \"faee1e90-99c1-4255-adb0-1d56308dbfd2\") " pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:12.965589 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964652    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f1aaf88-8ea0-4105-b62b-f784d91165d1-env-overrides\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.965589 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964678    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tpsm\" (UniqueName: \"kubernetes.io/projected/0835749c-5f54-425f-99de-161c4b66c81a-kube-api-access-2tpsm\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:12.965589 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964723    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-host-var-lib-cni-bin\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:12.965589 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964758    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-host-var-lib-kubelet\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:12.965589 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964794    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frkw7\" (UniqueName: \"kubernetes.io/projected/ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790-kube-api-access-frkw7\") pod \"node-ca-mlkk5\" (UID: \"ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790\") " pod="openshift-image-registry/node-ca-mlkk5"
Jul 11 23:31:12.965589 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.964833    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-modprobe-d\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.965589 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965026    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-modprobe-d\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.965589 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965065    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0835749c-5f54-425f-99de-161c4b66c81a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:12.965589 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965098    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-host-run-netns\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:12.966305 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965110    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9f1aaf88-8ea0-4105-b62b-f784d91165d1-env-overrides\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.966305 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965121    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-multus-conf-dir\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:12.966305 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965173    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.966305 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965195    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0835749c-5f54-425f-99de-161c4b66c81a-os-release\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:12.966305 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965219    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/43743170-d713-4977-a555-d59d7ab61f0c-cni-binary-copy\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:12.966305 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965240    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.966305 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965244    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-sysctl-conf\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.966305 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965289    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-var-lib-kubelet\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.966305 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965316    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-cni-netd\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.966305 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965341    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0835749c-5f54-425f-99de-161c4b66c81a-cnibin\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:12.966305 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965369    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-sysctl-conf\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.966305 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965385    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/89063ca1-d3ed-4f06-a51b-e10bc5035759-socket-dir\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:12.966305 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965386    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-var-lib-kubelet\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.966305 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965394    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-cni-netd\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.966305 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965411    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/faee1e90-99c1-4255-adb0-1d56308dbfd2-kubelet-config\") pod \"global-pull-secret-syncer-chgrx\" (UID: \"faee1e90-99c1-4255-adb0-1d56308dbfd2\") " pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:12.966305 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965469    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-host\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.966305 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965493    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-var-lib-openvswitch\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.967056 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965536    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-var-lib-openvswitch\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.967056 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965543    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-host\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.967056 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965562    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-log-socket\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.967056 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965590    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f1aaf88-8ea0-4105-b62b-f784d91165d1-ovn-node-metrics-cert\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.967056 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965610    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f1aaf88-8ea0-4105-b62b-f784d91165d1-ovnkube-script-lib\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.967056 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965623    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-log-socket\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.967056 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965644    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3680c2ef-7e40-4f46-b1aa-f3581c4f7626-iptables-alerter-script\") pod \"iptables-alerter-8j6d9\" (UID: \"3680c2ef-7e40-4f46-b1aa-f3581c4f7626\") " pod="openshift-network-operator/iptables-alerter-8j6d9"
Jul 11 23:31:12.967056 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965684    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3680c2ef-7e40-4f46-b1aa-f3581c4f7626-host-slash\") pod \"iptables-alerter-8j6d9\" (UID: \"3680c2ef-7e40-4f46-b1aa-f3581c4f7626\") " pod="openshift-network-operator/iptables-alerter-8j6d9"
Jul 11 23:31:12.967056 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965706    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4t27\" (UniqueName: \"kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27\") pod \"network-check-target-hrfff\" (UID: \"12be71b0-f3cc-4a5c-8109-03e7f70bc99f\") " pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:12.967056 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965736    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-systemd\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.967056 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965777    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nthhq\" (UniqueName: \"kubernetes.io/projected/19c676bb-ab21-4b12-8c3f-9183c5f222f7-kube-api-access-nthhq\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.967056 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965800    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-systemd\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.967056 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965806    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-kubelet\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.967056 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965849    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-kubelet\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.967056 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965859    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-slash\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.967056 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965879    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/89063ca1-d3ed-4f06-a51b-e10bc5035759-etc-selinux\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:12.967056 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965909    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-host-run-k8s-cni-cncf-io\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:12.967940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965951    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs\") pod \"network-metrics-daemon-5qc5j\" (UID: \"7b557007-8a5a-47eb-a23e-a96ed97fb64c\") " pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:12.967940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.965973    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790-host\") pod \"node-ca-mlkk5\" (UID: \"ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790\") " pod="openshift-image-registry/node-ca-mlkk5"
Jul 11 23:31:12.967940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966007    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-sysconfig\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.967940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966036    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-kubernetes\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.967940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966050    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790-host\") pod \"node-ca-mlkk5\" (UID: \"ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790\") " pod="openshift-image-registry/node-ca-mlkk5"
Jul 11 23:31:12.967940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966065    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-sys\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.967940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966090    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-run-openvswitch\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.967940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966097    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-slash\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.967940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966103    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-sysconfig\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.967940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966115    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-cnibin\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:12.967940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966143    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-sysctl-d\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.967940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966149    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-kubernetes\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.967940 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:12.966185    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 23:31:12.967940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966197    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-sys\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.967940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966198    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-run-openvswitch\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.967940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966226    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgjjb\" (UniqueName: \"kubernetes.io/projected/fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7-kube-api-access-vgjjb\") pod \"node-resolver-smg2m\" (UID: \"fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7\") " pod="openshift-dns/node-resolver-smg2m"
Jul 11 23:31:12.967940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966255    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0835749c-5f54-425f-99de-161c4b66c81a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:12.968779 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:12.966279    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs podName:7b557007-8a5a-47eb-a23e-a96ed97fb64c nodeName:}" failed. No retries permitted until 2026-07-11 23:31:13.466259426 +0000 UTC m=+3.044352195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs") pod "network-metrics-daemon-5qc5j" (UID: "7b557007-8a5a-47eb-a23e-a96ed97fb64c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 23:31:12.968779 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966286    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-sysctl-d\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.968779 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966339    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-multus-cni-dir\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:12.968779 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966363    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-multus-socket-dir-parent\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:12.968779 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966386    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-hostroot\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:12.968779 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966419    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19c676bb-ab21-4b12-8c3f-9183c5f222f7-tmp\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.968779 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966461    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-run-netns\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.968779 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966567    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9f1aaf88-8ea0-4105-b62b-f784d91165d1-host-run-netns\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.968779 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.966841    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9f1aaf88-8ea0-4105-b62b-f784d91165d1-ovnkube-script-lib\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.968779 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.968037    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/19c676bb-ab21-4b12-8c3f-9183c5f222f7-etc-tuned\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.968779 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.968623    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9f1aaf88-8ea0-4105-b62b-f784d91165d1-ovn-node-metrics-cert\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.969506 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.969486    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/19c676bb-ab21-4b12-8c3f-9183c5f222f7-tmp\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.974011 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.973854    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frkw7\" (UniqueName: \"kubernetes.io/projected/ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790-kube-api-access-frkw7\") pod \"node-ca-mlkk5\" (UID: \"ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790\") " pod="openshift-image-registry/node-ca-mlkk5"
Jul 11 23:31:12.974120 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.974015    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffdhf\" (UniqueName: \"kubernetes.io/projected/9f1aaf88-8ea0-4105-b62b-f784d91165d1-kube-api-access-ffdhf\") pod \"ovnkube-node-xd2gc\" (UID: \"9f1aaf88-8ea0-4105-b62b-f784d91165d1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:12.974227 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.974197    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nthhq\" (UniqueName: \"kubernetes.io/projected/19c676bb-ab21-4b12-8c3f-9183c5f222f7-kube-api-access-nthhq\") pod \"tuned-9vwnz\" (UID: \"19c676bb-ab21-4b12-8c3f-9183c5f222f7\") " pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:12.978290 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.975076    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bql6\" (UniqueName: \"kubernetes.io/projected/7b557007-8a5a-47eb-a23e-a96ed97fb64c-kube-api-access-6bql6\") pod \"network-metrics-daemon-5qc5j\" (UID: \"7b557007-8a5a-47eb-a23e-a96ed97fb64c\") " pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:12.979089 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.979061    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgjjb\" (UniqueName: \"kubernetes.io/projected/fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7-kube-api-access-vgjjb\") pod \"node-resolver-smg2m\" (UID: \"fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7\") " pod="openshift-dns/node-resolver-smg2m"
Jul 11 23:31:12.982207 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.981927    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal" event={"ID":"42968f545a222e19521d04bf195c5440","Type":"ContainerStarted","Data":"5a00d39318e53a689883605ccaac5894956ba5504f43b3f59366fa7591e4f6e6"}
Jul 11 23:31:12.983264 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:12.983241    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-83.ec2.internal" event={"ID":"6c9fb79e71283f9e071707f8e0898820","Type":"ContainerStarted","Data":"e5af311233f69b9538a090bfa5fc1b96429359d304c8e1ea851022494197da4a"}
Jul 11 23:31:13.067001 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.066968    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcd9p\" (UniqueName: \"kubernetes.io/projected/3680c2ef-7e40-4f46-b1aa-f3581c4f7626-kube-api-access-fcd9p\") pod \"iptables-alerter-8j6d9\" (UID: \"3680c2ef-7e40-4f46-b1aa-f3581c4f7626\") " pod="openshift-network-operator/iptables-alerter-8j6d9"
Jul 11 23:31:13.067165 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067010    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-host-var-lib-cni-multus\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.067165 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067034    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/43743170-d713-4977-a555-d59d7ab61f0c-multus-daemon-config\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.067165 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067102    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-host-var-lib-cni-multus\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.067165 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067155    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/89063ca1-d3ed-4f06-a51b-e10bc5035759-device-dir\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:13.067389 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067185    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/89063ca1-d3ed-4f06-a51b-e10bc5035759-sys-fs\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:13.067389 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067220    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-system-cni-dir\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.067389 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067243    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-os-release\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.067389 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067273    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0835749c-5f54-425f-99de-161c4b66c81a-system-cni-dir\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:13.067389 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067284    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/89063ca1-d3ed-4f06-a51b-e10bc5035759-device-dir\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:13.067389 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067296    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0835749c-5f54-425f-99de-161c4b66c81a-cni-binary-copy\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:13.067389 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067304    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/89063ca1-d3ed-4f06-a51b-e10bc5035759-sys-fs\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:13.067389 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067320    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret\") pod \"global-pull-secret-syncer-chgrx\" (UID: \"faee1e90-99c1-4255-adb0-1d56308dbfd2\") " pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:13.067389 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067328    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-os-release\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.067389 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067319    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-system-cni-dir\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.067389 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067354    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0835749c-5f54-425f-99de-161c4b66c81a-system-cni-dir\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:13.067389 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067346    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4ae214fa-346c-48ec-b3f7-67071c699d39-agent-certs\") pod \"konnectivity-agent-wh9hv\" (UID: \"4ae214fa-346c-48ec-b3f7-67071c699d39\") " pod="kube-system/konnectivity-agent-wh9hv"
Jul 11 23:31:13.067976 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067398    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4ae214fa-346c-48ec-b3f7-67071c699d39-konnectivity-ca\") pod \"konnectivity-agent-wh9hv\" (UID: \"4ae214fa-346c-48ec-b3f7-67071c699d39\") " pod="kube-system/konnectivity-agent-wh9hv"
Jul 11 23:31:13.067976 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:13.067408    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 11 23:31:13.067976 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067438    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-etc-kubernetes\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.067976 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067463    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/89063ca1-d3ed-4f06-a51b-e10bc5035759-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:13.067976 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:13.067484    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret podName:faee1e90-99c1-4255-adb0-1d56308dbfd2 nodeName:}" failed. No retries permitted until 2026-07-11 23:31:13.567464349 +0000 UTC m=+3.145557117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret") pod "global-pull-secret-syncer-chgrx" (UID: "faee1e90-99c1-4255-adb0-1d56308dbfd2") : object "kube-system"/"original-pull-secret" not registered
Jul 11 23:31:13.067976 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067503    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/89063ca1-d3ed-4f06-a51b-e10bc5035759-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:13.067976 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067519    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5wmp\" (UniqueName: \"kubernetes.io/projected/89063ca1-d3ed-4f06-a51b-e10bc5035759-kube-api-access-b5wmp\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:13.067976 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067541    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-etc-kubernetes\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.067976 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067547    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-host-run-multus-certs\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.067976 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067574    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9488l\" (UniqueName: \"kubernetes.io/projected/43743170-d713-4977-a555-d59d7ab61f0c-kube-api-access-9488l\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.067976 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067601    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0835749c-5f54-425f-99de-161c4b66c81a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:13.067976 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067626    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/89063ca1-d3ed-4f06-a51b-e10bc5035759-registration-dir\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:13.067976 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067642    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/43743170-d713-4977-a555-d59d7ab61f0c-multus-daemon-config\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.067976 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067650    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/faee1e90-99c1-4255-adb0-1d56308dbfd2-dbus\") pod \"global-pull-secret-syncer-chgrx\" (UID: \"faee1e90-99c1-4255-adb0-1d56308dbfd2\") " pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:13.067976 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067676    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tpsm\" (UniqueName: \"kubernetes.io/projected/0835749c-5f54-425f-99de-161c4b66c81a-kube-api-access-2tpsm\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:13.067976 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067703    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-host-var-lib-cni-bin\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.067976 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067719    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/89063ca1-d3ed-4f06-a51b-e10bc5035759-registration-dir\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:13.068796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067734    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-host-run-multus-certs\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.068796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067767    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-host-var-lib-kubelet\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.068796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067797    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0835749c-5f54-425f-99de-161c4b66c81a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:13.068796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067824    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-host-run-netns\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.068796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067847    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-multus-conf-dir\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.068796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067851    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0835749c-5f54-425f-99de-161c4b66c81a-cni-binary-copy\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:13.068796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067873    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0835749c-5f54-425f-99de-161c4b66c81a-os-release\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:13.068796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067896    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/43743170-d713-4977-a555-d59d7ab61f0c-cni-binary-copy\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.068796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067957    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0835749c-5f54-425f-99de-161c4b66c81a-cnibin\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:13.068796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067982    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/89063ca1-d3ed-4f06-a51b-e10bc5035759-socket-dir\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:13.068796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.067990    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4ae214fa-346c-48ec-b3f7-67071c699d39-konnectivity-ca\") pod \"konnectivity-agent-wh9hv\" (UID: \"4ae214fa-346c-48ec-b3f7-67071c699d39\") " pod="kube-system/konnectivity-agent-wh9hv"
Jul 11 23:31:13.068796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068007    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/faee1e90-99c1-4255-adb0-1d56308dbfd2-kubelet-config\") pod \"global-pull-secret-syncer-chgrx\" (UID: \"faee1e90-99c1-4255-adb0-1d56308dbfd2\") " pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:13.068796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068016    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0835749c-5f54-425f-99de-161c4b66c81a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:13.068796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068018    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-host-var-lib-cni-bin\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.068796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068047    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3680c2ef-7e40-4f46-b1aa-f3581c4f7626-iptables-alerter-script\") pod \"iptables-alerter-8j6d9\" (UID: \"3680c2ef-7e40-4f46-b1aa-f3581c4f7626\") " pod="openshift-network-operator/iptables-alerter-8j6d9"
Jul 11 23:31:13.068796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068050    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/faee1e90-99c1-4255-adb0-1d56308dbfd2-dbus\") pod \"global-pull-secret-syncer-chgrx\" (UID: \"faee1e90-99c1-4255-adb0-1d56308dbfd2\") " pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:13.068796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068078    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-host-var-lib-kubelet\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.069384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068074    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3680c2ef-7e40-4f46-b1aa-f3581c4f7626-host-slash\") pod \"iptables-alerter-8j6d9\" (UID: \"3680c2ef-7e40-4f46-b1aa-f3581c4f7626\") " pod="openshift-network-operator/iptables-alerter-8j6d9"
Jul 11 23:31:13.069384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068117    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4t27\" (UniqueName: \"kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27\") pod \"network-check-target-hrfff\" (UID: \"12be71b0-f3cc-4a5c-8109-03e7f70bc99f\") " pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:13.069384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068161    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/89063ca1-d3ed-4f06-a51b-e10bc5035759-etc-selinux\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:13.069384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068172    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/89063ca1-d3ed-4f06-a51b-e10bc5035759-socket-dir\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:13.069384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068185    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-host-run-k8s-cni-cncf-io\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.069384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068216    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0835749c-5f54-425f-99de-161c4b66c81a-cnibin\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:13.069384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068228    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-cnibin\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.069384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068132    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/faee1e90-99c1-4255-adb0-1d56308dbfd2-kubelet-config\") pod \"global-pull-secret-syncer-chgrx\" (UID: \"faee1e90-99c1-4255-adb0-1d56308dbfd2\") " pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:13.069384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068236    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-multus-conf-dir\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.069384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068261    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0835749c-5f54-425f-99de-161c4b66c81a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:13.069384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068276    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0835749c-5f54-425f-99de-161c4b66c81a-os-release\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:13.069384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068285    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-multus-cni-dir\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.069384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068279    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-host-run-netns\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.069384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068293    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3680c2ef-7e40-4f46-b1aa-f3581c4f7626-host-slash\") pod \"iptables-alerter-8j6d9\" (UID: \"3680c2ef-7e40-4f46-b1aa-f3581c4f7626\") " pod="openshift-network-operator/iptables-alerter-8j6d9"
Jul 11 23:31:13.069384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068311    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-multus-socket-dir-parent\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.069384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068326    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-cnibin\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.069384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068333    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-hostroot\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.069384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068371    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-host-run-k8s-cni-cncf-io\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.070088 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068383    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/89063ca1-d3ed-4f06-a51b-e10bc5035759-etc-selinux\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:13.070088 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068227    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0835749c-5f54-425f-99de-161c4b66c81a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:13.070088 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068532    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/43743170-d713-4977-a555-d59d7ab61f0c-cni-binary-copy\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.070088 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068575    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-multus-socket-dir-parent\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.070088 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068600    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-multus-cni-dir\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.070088 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068612    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/43743170-d713-4977-a555-d59d7ab61f0c-hostroot\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.070088 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.068640    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3680c2ef-7e40-4f46-b1aa-f3581c4f7626-iptables-alerter-script\") pod \"iptables-alerter-8j6d9\" (UID: \"3680c2ef-7e40-4f46-b1aa-f3581c4f7626\") " pod="openshift-network-operator/iptables-alerter-8j6d9"
Jul 11 23:31:13.070088 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.069566    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0835749c-5f54-425f-99de-161c4b66c81a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:13.070088 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.069999    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4ae214fa-346c-48ec-b3f7-67071c699d39-agent-certs\") pod \"konnectivity-agent-wh9hv\" (UID: \"4ae214fa-346c-48ec-b3f7-67071c699d39\") " pod="kube-system/konnectivity-agent-wh9hv"
Jul 11 23:31:13.074007 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:13.073986    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 11 23:31:13.074111 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:13.074013    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 11 23:31:13.074111 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:13.074027    2569 projected.go:196] Error preparing data for projected volume kube-api-access-r4t27 for pod openshift-network-diagnostics/network-check-target-hrfff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 23:31:13.074111 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:13.074092    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27 podName:12be71b0-f3cc-4a5c-8109-03e7f70bc99f nodeName:}" failed. No retries permitted until 2026-07-11 23:31:13.57407422 +0000 UTC m=+3.152166974 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r4t27" (UniqueName: "kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27") pod "network-check-target-hrfff" (UID: "12be71b0-f3cc-4a5c-8109-03e7f70bc99f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 23:31:13.075737 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.075713    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcd9p\" (UniqueName: \"kubernetes.io/projected/3680c2ef-7e40-4f46-b1aa-f3581c4f7626-kube-api-access-fcd9p\") pod \"iptables-alerter-8j6d9\" (UID: \"3680c2ef-7e40-4f46-b1aa-f3581c4f7626\") " pod="openshift-network-operator/iptables-alerter-8j6d9"
Jul 11 23:31:13.076341 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.076321    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5wmp\" (UniqueName: \"kubernetes.io/projected/89063ca1-d3ed-4f06-a51b-e10bc5035759-kube-api-access-b5wmp\") pod \"aws-ebs-csi-driver-node-x77ds\" (UID: \"89063ca1-d3ed-4f06-a51b-e10bc5035759\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:13.076665 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.076634    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9488l\" (UniqueName: \"kubernetes.io/projected/43743170-d713-4977-a555-d59d7ab61f0c-kube-api-access-9488l\") pod \"multus-nrn9c\" (UID: \"43743170-d713-4977-a555-d59d7ab61f0c\") " pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.076771 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.076726    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tpsm\" (UniqueName: \"kubernetes.io/projected/0835749c-5f54-425f-99de-161c4b66c81a-kube-api-access-2tpsm\") pod \"multus-additional-cni-plugins-jwbv4\" (UID: \"0835749c-5f54-425f-99de-161c4b66c81a\") " pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:13.106357 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.106280    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 11 23:31:13.157984 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.157955    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mlkk5"
Jul 11 23:31:13.167574 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.167551    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9vwnz"
Jul 11 23:31:13.177173 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.177152    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-smg2m"
Jul 11 23:31:13.182708 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.182691    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:13.190941 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.190920    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jwbv4"
Jul 11 23:31:13.197739 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.197721    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wh9hv"
Jul 11 23:31:13.206230 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.206210    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8j6d9"
Jul 11 23:31:13.213816 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.213797    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds"
Jul 11 23:31:13.220507 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.220483    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nrn9c"
Jul 11 23:31:13.471723 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.471636    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs\") pod \"network-metrics-daemon-5qc5j\" (UID: \"7b557007-8a5a-47eb-a23e-a96ed97fb64c\") " pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:13.471887 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:13.471803    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 23:31:13.471887 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:13.471875    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs podName:7b557007-8a5a-47eb-a23e-a96ed97fb64c nodeName:}" failed. No retries permitted until 2026-07-11 23:31:14.47185441 +0000 UTC m=+4.049947177 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs") pod "network-metrics-daemon-5qc5j" (UID: "7b557007-8a5a-47eb-a23e-a96ed97fb64c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 23:31:13.572968 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.572937    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret\") pod \"global-pull-secret-syncer-chgrx\" (UID: \"faee1e90-99c1-4255-adb0-1d56308dbfd2\") " pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:13.573122 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:13.573071    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 11 23:31:13.573165 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:13.573144    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret podName:faee1e90-99c1-4255-adb0-1d56308dbfd2 nodeName:}" failed. No retries permitted until 2026-07-11 23:31:14.573124707 +0000 UTC m=+4.151217468 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret") pod "global-pull-secret-syncer-chgrx" (UID: "faee1e90-99c1-4255-adb0-1d56308dbfd2") : object "kube-system"/"original-pull-secret" not registered
Jul 11 23:31:13.674123 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.674071    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4t27\" (UniqueName: \"kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27\") pod \"network-check-target-hrfff\" (UID: \"12be71b0-f3cc-4a5c-8109-03e7f70bc99f\") " pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:13.674284 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:13.674221    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 11 23:31:13.674284 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:13.674240    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 11 23:31:13.674284 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:13.674253    2569 projected.go:196] Error preparing data for projected volume kube-api-access-r4t27 for pod openshift-network-diagnostics/network-check-target-hrfff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 23:31:13.674403 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:13.674312    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27 podName:12be71b0-f3cc-4a5c-8109-03e7f70bc99f nodeName:}" failed. No retries permitted until 2026-07-11 23:31:14.674293358 +0000 UTC m=+4.252386127 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-r4t27" (UniqueName: "kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27") pod "network-check-target-hrfff" (UID: "12be71b0-f3cc-4a5c-8109-03e7f70bc99f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 23:31:13.890212 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.890170    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-10 23:26:12 +0000 UTC" deadline="2027-12-22 22:53:48.246271263 +0000 UTC"
Jul 11 23:31:13.890212 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.890209    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12695h22m34.356066631s"
Jul 11 23:31:13.893474 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:13.893438    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f1aaf88_8ea0_4105_b62b_f784d91165d1.slice/crio-7561a38181a64232b2e0104f4f917da9b79c729bc671a9788d6b83cd0e558916 WatchSource:0}: Error finding container 7561a38181a64232b2e0104f4f917da9b79c729bc671a9788d6b83cd0e558916: Status 404 returned error can't find the container with id 7561a38181a64232b2e0104f4f917da9b79c729bc671a9788d6b83cd0e558916
Jul 11 23:31:13.895690 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:13.895663    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0835749c_5f54_425f_99de_161c4b66c81a.slice/crio-29be8d4380b2735c468f524b47827fa66213057764574a14db35fd2bca76c195 WatchSource:0}: Error finding container 29be8d4380b2735c468f524b47827fa66213057764574a14db35fd2bca76c195: Status 404 returned error can't find the container with id 29be8d4380b2735c468f524b47827fa66213057764574a14db35fd2bca76c195
Jul 11 23:31:13.898298 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:13.898270    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc9214a0_6ad1_4315_a8d4_27b2f0a8d3a7.slice/crio-29bdd32ae9a200ac996bc374755f94e03ea59a26a551dbced00d1cba391f2c61 WatchSource:0}: Error finding container 29bdd32ae9a200ac996bc374755f94e03ea59a26a551dbced00d1cba391f2c61: Status 404 returned error can't find the container with id 29bdd32ae9a200ac996bc374755f94e03ea59a26a551dbced00d1cba391f2c61
Jul 11 23:31:13.900525 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:13.900093    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffcdbb3a_aa6c_4986_9a7f_fb26d0c8a790.slice/crio-46c8451d63b3b3667781b28fb48d6743f49fb47005fac6885a2c6aacbc9c14de WatchSource:0}: Error finding container 46c8451d63b3b3667781b28fb48d6743f49fb47005fac6885a2c6aacbc9c14de: Status 404 returned error can't find the container with id 46c8451d63b3b3667781b28fb48d6743f49fb47005fac6885a2c6aacbc9c14de
Jul 11 23:31:13.900902 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:13.900879    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89063ca1_d3ed_4f06_a51b_e10bc5035759.slice/crio-8c197da206470d7c7212fada417f7643ee87abedf4b9dfb1cf675518dc6729fd WatchSource:0}: Error finding container 8c197da206470d7c7212fada417f7643ee87abedf4b9dfb1cf675518dc6729fd: Status 404 returned error can't find the container with id 8c197da206470d7c7212fada417f7643ee87abedf4b9dfb1cf675518dc6729fd
Jul 11 23:31:13.902331 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:13.901345    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43743170_d713_4977_a555_d59d7ab61f0c.slice/crio-2da62723599e647469e9c5f8dddd540a91b3432d41e5ecf6f108218712abeefa WatchSource:0}: Error finding container 2da62723599e647469e9c5f8dddd540a91b3432d41e5ecf6f108218712abeefa: Status 404 returned error can't find the container with id 2da62723599e647469e9c5f8dddd540a91b3432d41e5ecf6f108218712abeefa
Jul 11 23:31:13.986491 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.986293    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nrn9c" event={"ID":"43743170-d713-4977-a555-d59d7ab61f0c","Type":"ContainerStarted","Data":"2da62723599e647469e9c5f8dddd540a91b3432d41e5ecf6f108218712abeefa"}
Jul 11 23:31:13.989230 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.989199    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wh9hv" event={"ID":"4ae214fa-346c-48ec-b3f7-67071c699d39","Type":"ContainerStarted","Data":"7c253ae3ac8a43fa036187160ab912184e1f8146bb6650ba84654e95d4e020c2"}
Jul 11 23:31:13.991824 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.991792    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jwbv4" event={"ID":"0835749c-5f54-425f-99de-161c4b66c81a","Type":"ContainerStarted","Data":"29be8d4380b2735c468f524b47827fa66213057764574a14db35fd2bca76c195"}
Jul 11 23:31:13.995202 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.995178    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc" event={"ID":"9f1aaf88-8ea0-4105-b62b-f784d91165d1","Type":"ContainerStarted","Data":"7561a38181a64232b2e0104f4f917da9b79c729bc671a9788d6b83cd0e558916"}
Jul 11 23:31:13.998996 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:13.998972    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9vwnz" event={"ID":"19c676bb-ab21-4b12-8c3f-9183c5f222f7","Type":"ContainerStarted","Data":"184bd6646b2a27e32e9c555817e28ac82f25de32bd4af43ce61d351d76bd9814"}
Jul 11 23:31:14.002132 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:14.002109    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds" event={"ID":"89063ca1-d3ed-4f06-a51b-e10bc5035759","Type":"ContainerStarted","Data":"8c197da206470d7c7212fada417f7643ee87abedf4b9dfb1cf675518dc6729fd"}
Jul 11 23:31:14.003132 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:14.003111    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mlkk5" event={"ID":"ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790","Type":"ContainerStarted","Data":"46c8451d63b3b3667781b28fb48d6743f49fb47005fac6885a2c6aacbc9c14de"}
Jul 11 23:31:14.004266 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:14.004246    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-smg2m" event={"ID":"fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7","Type":"ContainerStarted","Data":"29bdd32ae9a200ac996bc374755f94e03ea59a26a551dbced00d1cba391f2c61"}
Jul 11 23:31:14.010312 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:14.010288    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8j6d9" event={"ID":"3680c2ef-7e40-4f46-b1aa-f3581c4f7626","Type":"ContainerStarted","Data":"c962184c87297549879bae5dbecc696c7a27525f8822d144de6f566da513c97b"}
Jul 11 23:31:14.018338 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:14.018301    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-83.ec2.internal" podStartSLOduration=2.018290654 podStartE2EDuration="2.018290654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:14.018179406 +0000 UTC m=+3.596272177" watchObservedRunningTime="2026-07-11 23:31:14.018290654 +0000 UTC m=+3.596383430"
Jul 11 23:31:14.481859 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:14.481764    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs\") pod \"network-metrics-daemon-5qc5j\" (UID: \"7b557007-8a5a-47eb-a23e-a96ed97fb64c\") " pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:14.482016 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:14.481934    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 23:31:14.482016 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:14.482014    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs podName:7b557007-8a5a-47eb-a23e-a96ed97fb64c nodeName:}" failed. No retries permitted until 2026-07-11 23:31:16.481991024 +0000 UTC m=+6.060083798 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs") pod "network-metrics-daemon-5qc5j" (UID: "7b557007-8a5a-47eb-a23e-a96ed97fb64c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 23:31:14.582847 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:14.582803    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret\") pod \"global-pull-secret-syncer-chgrx\" (UID: \"faee1e90-99c1-4255-adb0-1d56308dbfd2\") " pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:14.583013 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:14.582965    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 11 23:31:14.583095 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:14.583029    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret podName:faee1e90-99c1-4255-adb0-1d56308dbfd2 nodeName:}" failed. No retries permitted until 2026-07-11 23:31:16.58301122 +0000 UTC m=+6.161103977 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret") pod "global-pull-secret-syncer-chgrx" (UID: "faee1e90-99c1-4255-adb0-1d56308dbfd2") : object "kube-system"/"original-pull-secret" not registered
Jul 11 23:31:14.683847 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:14.683808    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4t27\" (UniqueName: \"kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27\") pod \"network-check-target-hrfff\" (UID: \"12be71b0-f3cc-4a5c-8109-03e7f70bc99f\") " pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:14.684041 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:14.683990    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 11 23:31:14.684041 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:14.684012    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 11 23:31:14.684041 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:14.684025    2569 projected.go:196] Error preparing data for projected volume kube-api-access-r4t27 for pod openshift-network-diagnostics/network-check-target-hrfff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 23:31:14.684213 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:14.684082    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27 podName:12be71b0-f3cc-4a5c-8109-03e7f70bc99f nodeName:}" failed. No retries permitted until 2026-07-11 23:31:16.684063802 +0000 UTC m=+6.262156571 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-r4t27" (UniqueName: "kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27") pod "network-check-target-hrfff" (UID: "12be71b0-f3cc-4a5c-8109-03e7f70bc99f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 23:31:14.890968 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:14.890858    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-10 23:26:12 +0000 UTC" deadline="2027-12-15 03:36:06.377307934 +0000 UTC"
Jul 11 23:31:14.890968 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:14.890918    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12508h4m51.486394183s"
Jul 11 23:31:14.977041 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:14.977008    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:14.977206 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:14.977162    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5qc5j" podUID="7b557007-8a5a-47eb-a23e-a96ed97fb64c"
Jul 11 23:31:14.977606 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:14.977578    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:14.977737 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:14.977714    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chgrx" podUID="faee1e90-99c1-4255-adb0-1d56308dbfd2"
Jul 11 23:31:14.977876 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:14.977861    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:14.977961 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:14.977944    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrfff" podUID="12be71b0-f3cc-4a5c-8109-03e7f70bc99f"
Jul 11 23:31:15.053028 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:15.052988    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-83.ec2.internal" event={"ID":"6c9fb79e71283f9e071707f8e0898820","Type":"ContainerStarted","Data":"35bce62f3073ab97c8b87102eb666cd8b68691338f703a4db447885bd93675b6"}
Jul 11 23:31:15.063942 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:15.063902    2569 generic.go:356] "Generic (PLEG): container finished" podID="42968f545a222e19521d04bf195c5440" containerID="4c53ce277035e92e59118976d3c86604d4c2de614ce2e18b54a5024974f8e403" exitCode=0
Jul 11 23:31:15.064113 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:15.063953    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal" event={"ID":"42968f545a222e19521d04bf195c5440","Type":"ContainerDied","Data":"4c53ce277035e92e59118976d3c86604d4c2de614ce2e18b54a5024974f8e403"}
Jul 11 23:31:16.082214 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:16.081530    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal" event={"ID":"42968f545a222e19521d04bf195c5440","Type":"ContainerStarted","Data":"d32736c977ab0bc725523ce79945e1859e243b7fda6ccc0ada6cdc9af2ece85c"}
Jul 11 23:31:16.497991 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:16.497910    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs\") pod \"network-metrics-daemon-5qc5j\" (UID: \"7b557007-8a5a-47eb-a23e-a96ed97fb64c\") " pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:16.498188 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:16.498109    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 23:31:16.498188 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:16.498169    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs podName:7b557007-8a5a-47eb-a23e-a96ed97fb64c nodeName:}" failed. No retries permitted until 2026-07-11 23:31:20.498151181 +0000 UTC m=+10.076243939 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs") pod "network-metrics-daemon-5qc5j" (UID: "7b557007-8a5a-47eb-a23e-a96ed97fb64c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 23:31:16.599585 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:16.599033    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret\") pod \"global-pull-secret-syncer-chgrx\" (UID: \"faee1e90-99c1-4255-adb0-1d56308dbfd2\") " pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:16.599585 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:16.599190    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 11 23:31:16.599585 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:16.599252    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret podName:faee1e90-99c1-4255-adb0-1d56308dbfd2 nodeName:}" failed. No retries permitted until 2026-07-11 23:31:20.599234058 +0000 UTC m=+10.177326835 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret") pod "global-pull-secret-syncer-chgrx" (UID: "faee1e90-99c1-4255-adb0-1d56308dbfd2") : object "kube-system"/"original-pull-secret" not registered
Jul 11 23:31:16.701017 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:16.700372    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4t27\" (UniqueName: \"kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27\") pod \"network-check-target-hrfff\" (UID: \"12be71b0-f3cc-4a5c-8109-03e7f70bc99f\") " pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:16.701017 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:16.700605    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 11 23:31:16.701017 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:16.700627    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 11 23:31:16.701017 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:16.700639    2569 projected.go:196] Error preparing data for projected volume kube-api-access-r4t27 for pod openshift-network-diagnostics/network-check-target-hrfff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 23:31:16.701017 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:16.700699    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27 podName:12be71b0-f3cc-4a5c-8109-03e7f70bc99f nodeName:}" failed. No retries permitted until 2026-07-11 23:31:20.700680827 +0000 UTC m=+10.278773583 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-r4t27" (UniqueName: "kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27") pod "network-check-target-hrfff" (UID: "12be71b0-f3cc-4a5c-8109-03e7f70bc99f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 23:31:16.977552 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:16.976949    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:16.977552 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:16.977124    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5qc5j" podUID="7b557007-8a5a-47eb-a23e-a96ed97fb64c"
Jul 11 23:31:16.977552 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:16.977534    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:16.977847 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:16.977655    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chgrx" podUID="faee1e90-99c1-4255-adb0-1d56308dbfd2"
Jul 11 23:31:16.977847 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:16.977688    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:16.977847 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:16.977754    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrfff" podUID="12be71b0-f3cc-4a5c-8109-03e7f70bc99f"
Jul 11 23:31:18.977378 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:18.977343    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:18.977811 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:18.977475    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chgrx" podUID="faee1e90-99c1-4255-adb0-1d56308dbfd2"
Jul 11 23:31:18.977811 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:18.977350    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:18.977811 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:18.977660    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:18.977811 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:18.977752    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrfff" podUID="12be71b0-f3cc-4a5c-8109-03e7f70bc99f"
Jul 11 23:31:18.978021 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:18.977851    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5qc5j" podUID="7b557007-8a5a-47eb-a23e-a96ed97fb64c"
Jul 11 23:31:20.535771 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:20.535733    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs\") pod \"network-metrics-daemon-5qc5j\" (UID: \"7b557007-8a5a-47eb-a23e-a96ed97fb64c\") " pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:20.536248 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:20.535925    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 23:31:20.536248 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:20.536019    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs podName:7b557007-8a5a-47eb-a23e-a96ed97fb64c nodeName:}" failed. No retries permitted until 2026-07-11 23:31:28.535996778 +0000 UTC m=+18.114089552 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs") pod "network-metrics-daemon-5qc5j" (UID: "7b557007-8a5a-47eb-a23e-a96ed97fb64c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 23:31:20.636541 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:20.636346    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret\") pod \"global-pull-secret-syncer-chgrx\" (UID: \"faee1e90-99c1-4255-adb0-1d56308dbfd2\") " pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:20.636743 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:20.636568    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 11 23:31:20.636743 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:20.636653    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret podName:faee1e90-99c1-4255-adb0-1d56308dbfd2 nodeName:}" failed. No retries permitted until 2026-07-11 23:31:28.636629475 +0000 UTC m=+18.214722244 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret") pod "global-pull-secret-syncer-chgrx" (UID: "faee1e90-99c1-4255-adb0-1d56308dbfd2") : object "kube-system"/"original-pull-secret" not registered
Jul 11 23:31:20.737787 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:20.737750    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4t27\" (UniqueName: \"kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27\") pod \"network-check-target-hrfff\" (UID: \"12be71b0-f3cc-4a5c-8109-03e7f70bc99f\") " pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:20.737959 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:20.737944    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 11 23:31:20.738031 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:20.737971    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 11 23:31:20.738031 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:20.737987    2569 projected.go:196] Error preparing data for projected volume kube-api-access-r4t27 for pod openshift-network-diagnostics/network-check-target-hrfff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 23:31:20.738128 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:20.738056    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27 podName:12be71b0-f3cc-4a5c-8109-03e7f70bc99f nodeName:}" failed. No retries permitted until 2026-07-11 23:31:28.738036494 +0000 UTC m=+18.316129262 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-r4t27" (UniqueName: "kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27") pod "network-check-target-hrfff" (UID: "12be71b0-f3cc-4a5c-8109-03e7f70bc99f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 23:31:20.977958 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:20.977870    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:20.978111 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:20.977991    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5qc5j" podUID="7b557007-8a5a-47eb-a23e-a96ed97fb64c"
Jul 11 23:31:20.978111 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:20.978062    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:20.978213 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:20.978181    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chgrx" podUID="faee1e90-99c1-4255-adb0-1d56308dbfd2"
Jul 11 23:31:20.978257 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:20.978222    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:20.978306 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:20.978290    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrfff" podUID="12be71b0-f3cc-4a5c-8109-03e7f70bc99f"
Jul 11 23:31:22.976982 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:22.976944    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:22.977448 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:22.976944    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:22.977448 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:22.977075    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chgrx" podUID="faee1e90-99c1-4255-adb0-1d56308dbfd2"
Jul 11 23:31:22.977448 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:22.977086    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:22.977448 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:22.977184    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5qc5j" podUID="7b557007-8a5a-47eb-a23e-a96ed97fb64c"
Jul 11 23:31:22.977448 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:22.977276    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrfff" podUID="12be71b0-f3cc-4a5c-8109-03e7f70bc99f"
Jul 11 23:31:24.977548 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:24.977510    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:24.977994 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:24.977515    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:24.977994 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:24.977644    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrfff" podUID="12be71b0-f3cc-4a5c-8109-03e7f70bc99f"
Jul 11 23:31:24.977994 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:24.977519    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:24.977994 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:24.977722    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chgrx" podUID="faee1e90-99c1-4255-adb0-1d56308dbfd2"
Jul 11 23:31:24.977994 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:24.977829    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5qc5j" podUID="7b557007-8a5a-47eb-a23e-a96ed97fb64c"
Jul 11 23:31:26.977311 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:26.977263    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:26.977882 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:26.977326    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:26.977882 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:26.977340    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:26.977882 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:26.977463    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chgrx" podUID="faee1e90-99c1-4255-adb0-1d56308dbfd2"
Jul 11 23:31:26.977882 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:26.977515    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrfff" podUID="12be71b0-f3cc-4a5c-8109-03e7f70bc99f"
Jul 11 23:31:26.977882 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:26.977629    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5qc5j" podUID="7b557007-8a5a-47eb-a23e-a96ed97fb64c"
Jul 11 23:31:28.599220 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:28.599131    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs\") pod \"network-metrics-daemon-5qc5j\" (UID: \"7b557007-8a5a-47eb-a23e-a96ed97fb64c\") " pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:28.599705 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:28.599249    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 23:31:28.599705 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:28.599310    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs podName:7b557007-8a5a-47eb-a23e-a96ed97fb64c nodeName:}" failed. No retries permitted until 2026-07-11 23:31:44.599290822 +0000 UTC m=+34.177383577 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs") pod "network-metrics-daemon-5qc5j" (UID: "7b557007-8a5a-47eb-a23e-a96ed97fb64c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 23:31:28.700379 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:28.700336    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret\") pod \"global-pull-secret-syncer-chgrx\" (UID: \"faee1e90-99c1-4255-adb0-1d56308dbfd2\") " pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:28.700548 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:28.700531    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 11 23:31:28.700622 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:28.700611    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret podName:faee1e90-99c1-4255-adb0-1d56308dbfd2 nodeName:}" failed. No retries permitted until 2026-07-11 23:31:44.700590922 +0000 UTC m=+34.278683690 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret") pod "global-pull-secret-syncer-chgrx" (UID: "faee1e90-99c1-4255-adb0-1d56308dbfd2") : object "kube-system"/"original-pull-secret" not registered
Jul 11 23:31:28.801684 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:28.801635    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4t27\" (UniqueName: \"kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27\") pod \"network-check-target-hrfff\" (UID: \"12be71b0-f3cc-4a5c-8109-03e7f70bc99f\") " pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:28.801872 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:28.801834    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 11 23:31:28.801872 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:28.801865    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 11 23:31:28.801968 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:28.801878    2569 projected.go:196] Error preparing data for projected volume kube-api-access-r4t27 for pod openshift-network-diagnostics/network-check-target-hrfff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 23:31:28.801968 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:28.801952    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27 podName:12be71b0-f3cc-4a5c-8109-03e7f70bc99f nodeName:}" failed. No retries permitted until 2026-07-11 23:31:44.801930696 +0000 UTC m=+34.380023466 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-r4t27" (UniqueName: "kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27") pod "network-check-target-hrfff" (UID: "12be71b0-f3cc-4a5c-8109-03e7f70bc99f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 23:31:28.977520 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:28.977436    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:28.977667 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:28.977448    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:28.977667 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:28.977563    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:28.977667 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:28.977562    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrfff" podUID="12be71b0-f3cc-4a5c-8109-03e7f70bc99f"
Jul 11 23:31:28.977787 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:28.977702    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5qc5j" podUID="7b557007-8a5a-47eb-a23e-a96ed97fb64c"
Jul 11 23:31:28.977837 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:28.977813    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chgrx" podUID="faee1e90-99c1-4255-adb0-1d56308dbfd2"
Jul 11 23:31:30.978062 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:30.978027    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:30.978518 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:30.978106    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:30.978518 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:30.978132    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrfff" podUID="12be71b0-f3cc-4a5c-8109-03e7f70bc99f"
Jul 11 23:31:30.978518 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:30.978205    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5qc5j" podUID="7b557007-8a5a-47eb-a23e-a96ed97fb64c"
Jul 11 23:31:30.978518 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:30.978263    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:30.978518 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:30.978363    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chgrx" podUID="faee1e90-99c1-4255-adb0-1d56308dbfd2"
Jul 11 23:31:32.976930 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:32.976719    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:32.977292 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:32.976719    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:32.977292 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:32.976750    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:32.977292 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:32.977051    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chgrx" podUID="faee1e90-99c1-4255-adb0-1d56308dbfd2"
Jul 11 23:31:32.980038 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:32.979289    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrfff" podUID="12be71b0-f3cc-4a5c-8109-03e7f70bc99f"
Jul 11 23:31:32.982608 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:32.982574    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5qc5j" podUID="7b557007-8a5a-47eb-a23e-a96ed97fb64c"
Jul 11 23:31:33.122298 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:33.122253    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds" event={"ID":"89063ca1-d3ed-4f06-a51b-e10bc5035759","Type":"ContainerStarted","Data":"bd95ca75ddc603116429e4fcd1bea17eee0bd04c0f2562dfa034493f60acaa23"}
Jul 11 23:31:33.124018 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:33.123988    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mlkk5" event={"ID":"ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790","Type":"ContainerStarted","Data":"31443ee20ede3151ab482e759d302e7ac44bf79f2b180f64285dd2323aed94de"}
Jul 11 23:31:33.125848 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:33.125812    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-smg2m" event={"ID":"fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7","Type":"ContainerStarted","Data":"9005b8c788d2ba21682bd954083bb957f9a50305f9542d4d2b9db9fb020df97e"}
Jul 11 23:31:33.127418 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:33.127396    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nrn9c" event={"ID":"43743170-d713-4977-a555-d59d7ab61f0c","Type":"ContainerStarted","Data":"217a594aea43194c5feb2e2d1d257e7dfe31b21138f2559c0963925a5fd86bea"}
Jul 11 23:31:33.129256 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:33.129222    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wh9hv" event={"ID":"4ae214fa-346c-48ec-b3f7-67071c699d39","Type":"ContainerStarted","Data":"c3a129543e4c5cc2ab85bcdc3e4408c5489842733c7ccfeee166cfd84d1026d5"}
Jul 11 23:31:33.130852 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:33.130823    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jwbv4" event={"ID":"0835749c-5f54-425f-99de-161c4b66c81a","Type":"ContainerStarted","Data":"ba619d70a0c60477d97a325cfe1fce80b6f6b98b1adfdda3cedd121193db66ec"}
Jul 11 23:31:33.132520 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:33.132497    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9vwnz" event={"ID":"19c676bb-ab21-4b12-8c3f-9183c5f222f7","Type":"ContainerStarted","Data":"9041604747fda39ff5be2e9a235470488cd72f3c440223712c8fa84829cc7111"}
Jul 11 23:31:33.138136 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:33.138094    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-83.ec2.internal" podStartSLOduration=21.138081801 podStartE2EDuration="21.138081801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:16.098079982 +0000 UTC m=+5.676172761" watchObservedRunningTime="2026-07-11 23:31:33.138081801 +0000 UTC m=+22.716174577"
Jul 11 23:31:33.154374 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:33.154322    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mlkk5" podStartSLOduration=3.2532715469999998 podStartE2EDuration="22.154302011s" totalImagesPullingTime="18.901030464s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:33.137703453 +0000 UTC m=+22.715796240" watchObservedRunningTime="2026-07-11 23:31:33.154302011 +0000 UTC m=+22.732394791"
Jul 11 23:31:33.154510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:33.154461    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nrn9c" podStartSLOduration=3.235169047 podStartE2EDuration="22.154448748s" totalImagesPullingTime="18.919279701s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:33.153811745 +0000 UTC m=+22.731904522" watchObservedRunningTime="2026-07-11 23:31:33.154448748 +0000 UTC m=+22.732541515"
Jul 11 23:31:33.172414 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:33.172354    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9vwnz" podStartSLOduration=3.274543846 podStartE2EDuration="22.172333116s" totalImagesPullingTime="18.89778927s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:33.171357847 +0000 UTC m=+22.749450622" watchObservedRunningTime="2026-07-11 23:31:33.172333116 +0000 UTC m=+22.750425986"
Jul 11 23:31:33.206137 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:33.206093    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-smg2m" podStartSLOduration=3.303899264 podStartE2EDuration="22.206077276s" totalImagesPullingTime="18.902178012s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:33.205842717 +0000 UTC m=+22.783935493" watchObservedRunningTime="2026-07-11 23:31:33.206077276 +0000 UTC m=+22.784170055"
Jul 11 23:31:33.219825 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:33.219770    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wh9hv" podStartSLOduration=7.9944515240000005 podStartE2EDuration="22.219756284s" totalImagesPullingTime="14.22530476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:33.219460927 +0000 UTC m=+22.797553699" watchObservedRunningTime="2026-07-11 23:31:33.219756284 +0000 UTC m=+22.797849060"
Jul 11 23:31:34.135778 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.135590    2569 generic.go:356] "Generic (PLEG): container finished" podID="0835749c-5f54-425f-99de-161c4b66c81a" containerID="ba619d70a0c60477d97a325cfe1fce80b6f6b98b1adfdda3cedd121193db66ec" exitCode=0
Jul 11 23:31:34.136586 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.135672    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jwbv4" event={"ID":"0835749c-5f54-425f-99de-161c4b66c81a","Type":"ContainerDied","Data":"ba619d70a0c60477d97a325cfe1fce80b6f6b98b1adfdda3cedd121193db66ec"}
Jul 11 23:31:34.138414 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.138392    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 11 23:31:34.138737 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.138717    2569 generic.go:356] "Generic (PLEG): container finished" podID="9f1aaf88-8ea0-4105-b62b-f784d91165d1" containerID="bb12f76024c76ae0fda540a10a8de19c12f056e16509b89ddcf05e837465cb3e" exitCode=1
Jul 11 23:31:34.138833 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.138811    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc" event={"ID":"9f1aaf88-8ea0-4105-b62b-f784d91165d1","Type":"ContainerStarted","Data":"f35237bea88f8fb73a5a4de5406f068866a09148570e5711156a9de3e553c045"}
Jul 11 23:31:34.138930 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.138882    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc" event={"ID":"9f1aaf88-8ea0-4105-b62b-f784d91165d1","Type":"ContainerStarted","Data":"107a64b760e009dec15a50c6221dc9885e91253904fdb51f600f6024f146815a"}
Jul 11 23:31:34.138981 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.138954    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc" event={"ID":"9f1aaf88-8ea0-4105-b62b-f784d91165d1","Type":"ContainerStarted","Data":"d142086ad0aa310c83886cbfc5b435c0fd7dfec4ccfc9a7f4efc7f01c9500d54"}
Jul 11 23:31:34.139032 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.138994    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc" event={"ID":"9f1aaf88-8ea0-4105-b62b-f784d91165d1","Type":"ContainerStarted","Data":"66cb0947831424ecffc955836bb5ec7acbbbd46933e703381f0e0a86bc44bb32"}
Jul 11 23:31:34.139032 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.139024    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc" event={"ID":"9f1aaf88-8ea0-4105-b62b-f784d91165d1","Type":"ContainerDied","Data":"bb12f76024c76ae0fda540a10a8de19c12f056e16509b89ddcf05e837465cb3e"}
Jul 11 23:31:34.139135 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.139061    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc" event={"ID":"9f1aaf88-8ea0-4105-b62b-f784d91165d1","Type":"ContainerStarted","Data":"be8defee47a9338adf0a2e5909fd104ebd0baa366213a17576ebf830e089f233"}
Jul 11 23:31:34.276833 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.276811    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 11 23:31:34.595586 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.595513    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wh9hv"
Jul 11 23:31:34.596114 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.596096    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wh9hv"
Jul 11 23:31:34.927037 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.926880    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-11T23:31:34.276827656Z","UUID":"6bf65a32-9b09-45ab-bb34-11243acb495f","Handler":null,"Name":"","Endpoint":""}
Jul 11 23:31:34.928973 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.928571    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 11 23:31:34.928973 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.928605    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 11 23:31:34.977568 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.977540    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:34.977799 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:34.977672    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chgrx" podUID="faee1e90-99c1-4255-adb0-1d56308dbfd2"
Jul 11 23:31:34.977799 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.977709    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:34.977929 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:34.977804    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5qc5j" podUID="7b557007-8a5a-47eb-a23e-a96ed97fb64c"
Jul 11 23:31:34.977929 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:34.977552    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:34.977929 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:34.977880    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrfff" podUID="12be71b0-f3cc-4a5c-8109-03e7f70bc99f"
Jul 11 23:31:35.142700 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:35.142637    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8j6d9" event={"ID":"3680c2ef-7e40-4f46-b1aa-f3581c4f7626","Type":"ContainerStarted","Data":"f27d1cd0e27bdba93048c81754a9725be5c8082978de2ce6a50789086bc01427"}
Jul 11 23:31:35.144755 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:35.144710    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds" event={"ID":"89063ca1-d3ed-4f06-a51b-e10bc5035759","Type":"ContainerStarted","Data":"68c3affdac093466f4f5cb6646660e9199ef55bb3998f48c0d40063d7d0f7022"}
Jul 11 23:31:35.144891 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:35.144791    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wh9hv"
Jul 11 23:31:35.145349 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:35.145331    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wh9hv"
Jul 11 23:31:35.156752 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:35.156697    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8j6d9" podStartSLOduration=5.261348202 podStartE2EDuration="24.156683374s" totalImagesPullingTime="18.895335172s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:35.156529197 +0000 UTC m=+24.734621995" watchObservedRunningTime="2026-07-11 23:31:35.156683374 +0000 UTC m=+24.734776149"
Jul 11 23:31:36.150731 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:36.150515    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 11 23:31:36.151434 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:36.151389    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc" event={"ID":"9f1aaf88-8ea0-4105-b62b-f784d91165d1","Type":"ContainerStarted","Data":"d289f5ed748fdda1d42c5933c0ce7c88971e3417622cfafb9603965793a8bb54"}
Jul 11 23:31:36.153634 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:36.153605    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds" event={"ID":"89063ca1-d3ed-4f06-a51b-e10bc5035759","Type":"ContainerStarted","Data":"7d673d66f87c59581e7577db5c68b5f246ba981cc6bdf51ee85287bed9875c4d"}
Jul 11 23:31:36.170110 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:36.170050    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x77ds" podStartSLOduration=4.244735404 podStartE2EDuration="25.17002963s" totalImagesPullingTime="20.925294226s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:36.169651087 +0000 UTC m=+25.747743863" watchObservedRunningTime="2026-07-11 23:31:36.17002963 +0000 UTC m=+25.748122407"
Jul 11 23:31:36.977324 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:36.977289    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:36.977528 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:36.977289    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:36.977528 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:36.977289    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:36.977528 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:36.977459    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5qc5j" podUID="7b557007-8a5a-47eb-a23e-a96ed97fb64c"
Jul 11 23:31:36.977528 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:36.977511    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chgrx" podUID="faee1e90-99c1-4255-adb0-1d56308dbfd2"
Jul 11 23:31:36.977751 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:36.977603    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrfff" podUID="12be71b0-f3cc-4a5c-8109-03e7f70bc99f"
Jul 11 23:31:38.977768 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:38.977543    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:38.978485 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:38.977553    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:38.978485 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:38.977797    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chgrx" podUID="faee1e90-99c1-4255-adb0-1d56308dbfd2"
Jul 11 23:31:38.978485 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:38.977897    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrfff" podUID="12be71b0-f3cc-4a5c-8109-03e7f70bc99f"
Jul 11 23:31:38.978485 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:38.977559    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:38.978485 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:38.978011    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5qc5j" podUID="7b557007-8a5a-47eb-a23e-a96ed97fb64c"
Jul 11 23:31:39.161909 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:39.161874    2569 generic.go:356] "Generic (PLEG): container finished" podID="0835749c-5f54-425f-99de-161c4b66c81a" containerID="b7f031c1ee585a07f9e7dbfa21c6d79302dfcab7f079af1a852a60b90e1986f1" exitCode=0
Jul 11 23:31:39.162095 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:39.161972    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jwbv4" event={"ID":"0835749c-5f54-425f-99de-161c4b66c81a","Type":"ContainerDied","Data":"b7f031c1ee585a07f9e7dbfa21c6d79302dfcab7f079af1a852a60b90e1986f1"}
Jul 11 23:31:39.165505 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:39.165486    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 11 23:31:39.165851 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:39.165830    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc" event={"ID":"9f1aaf88-8ea0-4105-b62b-f784d91165d1","Type":"ContainerStarted","Data":"f61f0e12247cb12c4d1b0e7fc33b07a8f44ea05738e35c6a3c0385d69d41a4ec"}
Jul 11 23:31:39.166061 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:39.166046    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:39.166189 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:39.166176    2569 scope.go:117] "RemoveContainer" containerID="bb12f76024c76ae0fda540a10a8de19c12f056e16509b89ddcf05e837465cb3e"
Jul 11 23:31:39.181412 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:39.181390    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:40.170918 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:40.170778    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 11 23:31:40.171270 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:40.171207    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc" event={"ID":"9f1aaf88-8ea0-4105-b62b-f784d91165d1","Type":"ContainerStarted","Data":"8dd59f62a633d369703a5473d15331076a82a322e5cac8fa5453e3818448ca0c"}
Jul 11 23:31:40.171311 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:40.171293    2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 11 23:31:40.171556 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:40.171537    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:40.185809 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:40.185785    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:40.198513 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:40.198460    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc" podStartSLOduration=9.958013049 podStartE2EDuration="29.1984441s" totalImagesPullingTime="19.240431051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:40.196419992 +0000 UTC m=+29.774512773" watchObservedRunningTime="2026-07-11 23:31:40.1984441 +0000 UTC m=+29.776536867"
Jul 11 23:31:40.219142 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:40.219109    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-chgrx"]
Jul 11 23:31:40.219279 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:40.219267    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:40.219376 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:40.219356    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chgrx" podUID="faee1e90-99c1-4255-adb0-1d56308dbfd2"
Jul 11 23:31:40.222357 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:40.222328    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hrfff"]
Jul 11 23:31:40.222495 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:40.222468    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:40.222594 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:40.222571    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrfff" podUID="12be71b0-f3cc-4a5c-8109-03e7f70bc99f"
Jul 11 23:31:40.222885 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:40.222865    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5qc5j"]
Jul 11 23:31:40.222989 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:40.222975    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:40.223086 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:40.223068    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5qc5j" podUID="7b557007-8a5a-47eb-a23e-a96ed97fb64c"
Jul 11 23:31:41.175574 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:41.175542    2569 generic.go:356] "Generic (PLEG): container finished" podID="0835749c-5f54-425f-99de-161c4b66c81a" containerID="6c0c7f40cded695598454c8410d4178ea00dc03ca82b4d4a3a2368877b870f41" exitCode=0
Jul 11 23:31:41.176042 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:41.175627    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jwbv4" event={"ID":"0835749c-5f54-425f-99de-161c4b66c81a","Type":"ContainerDied","Data":"6c0c7f40cded695598454c8410d4178ea00dc03ca82b4d4a3a2368877b870f41"}
Jul 11 23:31:41.176042 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:41.175915    2569 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 11 23:31:41.596239 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:41.596169    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:31:41.977592 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:41.977563    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:41.977695 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:41.977667    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:41.977695 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:41.977675    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chgrx" podUID="faee1e90-99c1-4255-adb0-1d56308dbfd2"
Jul 11 23:31:41.977695 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:41.977695    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:41.977811 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:41.977790    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5qc5j" podUID="7b557007-8a5a-47eb-a23e-a96ed97fb64c"
Jul 11 23:31:41.977873 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:41.977856    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrfff" podUID="12be71b0-f3cc-4a5c-8109-03e7f70bc99f"
Jul 11 23:31:42.180254 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:42.180218    2569 generic.go:356] "Generic (PLEG): container finished" podID="0835749c-5f54-425f-99de-161c4b66c81a" containerID="fbcbddaf73787f351e349bf4f24144b4361e96098b995c942c8708073ca339f5" exitCode=0
Jul 11 23:31:42.180830 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:42.180306    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jwbv4" event={"ID":"0835749c-5f54-425f-99de-161c4b66c81a","Type":"ContainerDied","Data":"fbcbddaf73787f351e349bf4f24144b4361e96098b995c942c8708073ca339f5"}
Jul 11 23:31:43.196455 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:43.196349    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc" podUID="9f1aaf88-8ea0-4105-b62b-f784d91165d1" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 11 23:31:43.976808 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:43.976767    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:43.976808 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:43.976808    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:43.977053 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:43.976879    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:43.977053 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:43.976893    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chgrx" podUID="faee1e90-99c1-4255-adb0-1d56308dbfd2"
Jul 11 23:31:43.977053 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:43.977003    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5qc5j" podUID="7b557007-8a5a-47eb-a23e-a96ed97fb64c"
Jul 11 23:31:43.977227 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:43.977094    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrfff" podUID="12be71b0-f3cc-4a5c-8109-03e7f70bc99f"
Jul 11 23:31:44.621125 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:44.621086    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs\") pod \"network-metrics-daemon-5qc5j\" (UID: \"7b557007-8a5a-47eb-a23e-a96ed97fb64c\") " pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:44.621510 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:44.621222    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 23:31:44.621510 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:44.621295    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs podName:7b557007-8a5a-47eb-a23e-a96ed97fb64c nodeName:}" failed. No retries permitted until 2026-07-11 23:32:16.621273035 +0000 UTC m=+66.199365806 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs") pod "network-metrics-daemon-5qc5j" (UID: "7b557007-8a5a-47eb-a23e-a96ed97fb64c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 11 23:31:44.722166 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:44.722130    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret\") pod \"global-pull-secret-syncer-chgrx\" (UID: \"faee1e90-99c1-4255-adb0-1d56308dbfd2\") " pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:44.722354 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:44.722332    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 11 23:31:44.722446 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:44.722417    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret podName:faee1e90-99c1-4255-adb0-1d56308dbfd2 nodeName:}" failed. No retries permitted until 2026-07-11 23:32:16.722393971 +0000 UTC m=+66.300486741 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret") pod "global-pull-secret-syncer-chgrx" (UID: "faee1e90-99c1-4255-adb0-1d56308dbfd2") : object "kube-system"/"original-pull-secret" not registered
Jul 11 23:31:44.823197 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:44.823161    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4t27\" (UniqueName: \"kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27\") pod \"network-check-target-hrfff\" (UID: \"12be71b0-f3cc-4a5c-8109-03e7f70bc99f\") " pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:44.823354 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:44.823329    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 11 23:31:44.823420 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:44.823355    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 11 23:31:44.823420 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:44.823370    2569 projected.go:196] Error preparing data for projected volume kube-api-access-r4t27 for pod openshift-network-diagnostics/network-check-target-hrfff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 23:31:44.823524 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:44.823447    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27 podName:12be71b0-f3cc-4a5c-8109-03e7f70bc99f nodeName:}" failed. No retries permitted until 2026-07-11 23:32:16.823420635 +0000 UTC m=+66.401513389 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-r4t27" (UniqueName: "kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27") pod "network-check-target-hrfff" (UID: "12be71b0-f3cc-4a5c-8109-03e7f70bc99f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 11 23:31:45.977917 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:45.977692    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:45.978379 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:45.977698    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:45.978379 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:45.978019    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hrfff" podUID="12be71b0-f3cc-4a5c-8109-03e7f70bc99f"
Jul 11 23:31:45.978379 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:45.977709    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:45.978379 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:45.978095    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5qc5j" podUID="7b557007-8a5a-47eb-a23e-a96ed97fb64c"
Jul 11 23:31:45.978379 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:45.978148    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-chgrx" podUID="faee1e90-99c1-4255-adb0-1d56308dbfd2"
Jul 11 23:31:46.193180 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.193140    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jwbv4" event={"ID":"0835749c-5f54-425f-99de-161c4b66c81a","Type":"ContainerStarted","Data":"8183b65b9c8bf16189899e8c97fdf0cb069658a2b492b30c59dd43e9e726eeb4"}
Jul 11 23:31:46.697952 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.697877    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-83.ec2.internal" event="NodeReady"
Jul 11 23:31:46.698119 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.698008    2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 11 23:31:46.727632 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.727599    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f664cb986-z7cww"]
Jul 11 23:31:46.730547 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.730522    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.733222 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.733201    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-h9mt9\""
Jul 11 23:31:46.733329 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.733235    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 11 23:31:46.733329 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.733249    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 11 23:31:46.733329 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.733319    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 11 23:31:46.738071 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.738054    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 11 23:31:46.743028 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.742992    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-25dj7"]
Jul 11 23:31:46.745700 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.745682    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jvtd6"]
Jul 11 23:31:46.745818 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.745803    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-25dj7"
Jul 11 23:31:46.748187 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.748168    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8s5q6\""
Jul 11 23:31:46.748328 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.748305    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 11 23:31:46.748460 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.748395    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 11 23:31:46.748529 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.748494    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f664cb986-z7cww"]
Jul 11 23:31:46.748529 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.748508    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 11 23:31:46.748621 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.748581    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:31:46.751720 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.751702    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 11 23:31:46.751816 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.751755    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 11 23:31:46.751888 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.751837    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8jgcr\""
Jul 11 23:31:46.755534 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.755511    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-25dj7"]
Jul 11 23:31:46.762845 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.762275    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jvtd6"]
Jul 11 23:31:46.839585 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.839526    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c1dc75e-e9b9-4f38-916c-51230cc19a38-installation-pull-secrets\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.839585 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.839586    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/03eb4687-6b09-4c00-8601-4f7e11f8de4c-cert\") pod \"ingress-canary-25dj7\" (UID: \"03eb4687-6b09-4c00-8601-4f7e11f8de4c\") " pod="openshift-ingress-canary/ingress-canary-25dj7"
Jul 11 23:31:46.839798 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.839608    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsrhl\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-kube-api-access-zsrhl\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.839798 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.839627    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.839798 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.839643    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c1dc75e-e9b9-4f38-916c-51230cc19a38-trusted-ca\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.839798 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.839659    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-config-volume\") pod \"dns-default-jvtd6\" (UID: \"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf\") " pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:31:46.839798 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.839749    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lklrw\" (UniqueName: \"kubernetes.io/projected/03eb4687-6b09-4c00-8601-4f7e11f8de4c-kube-api-access-lklrw\") pod \"ingress-canary-25dj7\" (UID: \"03eb4687-6b09-4c00-8601-4f7e11f8de4c\") " pod="openshift-ingress-canary/ingress-canary-25dj7"
Jul 11 23:31:46.839945 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.839798    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c1dc75e-e9b9-4f38-916c-51230cc19a38-image-registry-private-configuration\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.839945 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.839816    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvnrl\" (UniqueName: \"kubernetes.io/projected/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-kube-api-access-cvnrl\") pod \"dns-default-jvtd6\" (UID: \"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf\") " pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:31:46.839945 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.839853    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-tmp-dir\") pod \"dns-default-jvtd6\" (UID: \"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf\") " pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:31:46.839945 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.839927    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c1dc75e-e9b9-4f38-916c-51230cc19a38-ca-trust-extracted\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.840061 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.839945    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-certificates\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.840061 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.839969    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-bound-sa-token\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.840061 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.839988    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-metrics-tls\") pod \"dns-default-jvtd6\" (UID: \"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf\") " pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:31:46.941146 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.941087    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c1dc75e-e9b9-4f38-916c-51230cc19a38-image-registry-private-configuration\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.941146 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.941136    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvnrl\" (UniqueName: \"kubernetes.io/projected/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-kube-api-access-cvnrl\") pod \"dns-default-jvtd6\" (UID: \"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf\") " pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:31:46.941146 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.941156    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-tmp-dir\") pod \"dns-default-jvtd6\" (UID: \"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf\") " pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:31:46.941453 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.941205    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c1dc75e-e9b9-4f38-916c-51230cc19a38-ca-trust-extracted\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.941453 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.941221    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-certificates\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.941453 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.941241    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-bound-sa-token\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.941453 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.941267    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-metrics-tls\") pod \"dns-default-jvtd6\" (UID: \"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf\") " pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:31:46.941453 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.941297    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c1dc75e-e9b9-4f38-916c-51230cc19a38-installation-pull-secrets\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.941453 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.941324    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/03eb4687-6b09-4c00-8601-4f7e11f8de4c-cert\") pod \"ingress-canary-25dj7\" (UID: \"03eb4687-6b09-4c00-8601-4f7e11f8de4c\") " pod="openshift-ingress-canary/ingress-canary-25dj7"
Jul 11 23:31:46.941453 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.941353    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsrhl\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-kube-api-access-zsrhl\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.941453 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.941378    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.941453 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.941398    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c1dc75e-e9b9-4f38-916c-51230cc19a38-trusted-ca\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.941453 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:46.941404    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 11 23:31:46.941453 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.941420    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-config-volume\") pod \"dns-default-jvtd6\" (UID: \"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf\") " pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:31:46.941943 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:46.941491    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-metrics-tls podName:b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf nodeName:}" failed. No retries permitted until 2026-07-11 23:31:47.441469062 +0000 UTC m=+37.019561839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-metrics-tls") pod "dns-default-jvtd6" (UID: "b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf") : secret "dns-default-metrics-tls" not found
Jul 11 23:31:46.941943 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.941512    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lklrw\" (UniqueName: \"kubernetes.io/projected/03eb4687-6b09-4c00-8601-4f7e11f8de4c-kube-api-access-lklrw\") pod \"ingress-canary-25dj7\" (UID: \"03eb4687-6b09-4c00-8601-4f7e11f8de4c\") " pod="openshift-ingress-canary/ingress-canary-25dj7"
Jul 11 23:31:46.941943 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:46.941601    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 11 23:31:46.941943 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:46.941620    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f664cb986-z7cww: secret "image-registry-tls" not found
Jul 11 23:31:46.941943 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.941645    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-tmp-dir\") pod \"dns-default-jvtd6\" (UID: \"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf\") " pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:31:46.941943 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:46.941676    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls podName:3c1dc75e-e9b9-4f38-916c-51230cc19a38 nodeName:}" failed. No retries permitted until 2026-07-11 23:31:47.441660491 +0000 UTC m=+37.019753262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls") pod "image-registry-7f664cb986-z7cww" (UID: "3c1dc75e-e9b9-4f38-916c-51230cc19a38") : secret "image-registry-tls" not found
Jul 11 23:31:46.942238 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.942126    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-config-volume\") pod \"dns-default-jvtd6\" (UID: \"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf\") " pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:31:46.942238 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.942179    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c1dc75e-e9b9-4f38-916c-51230cc19a38-ca-trust-extracted\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.942360 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.942337    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-certificates\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.942823 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.942799    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c1dc75e-e9b9-4f38-916c-51230cc19a38-trusted-ca\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.945383 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.945364    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c1dc75e-e9b9-4f38-916c-51230cc19a38-image-registry-private-configuration\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.945383 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.945373    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/03eb4687-6b09-4c00-8601-4f7e11f8de4c-cert\") pod \"ingress-canary-25dj7\" (UID: \"03eb4687-6b09-4c00-8601-4f7e11f8de4c\") " pod="openshift-ingress-canary/ingress-canary-25dj7"
Jul 11 23:31:46.945509 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.945466    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c1dc75e-e9b9-4f38-916c-51230cc19a38-installation-pull-secrets\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.949508 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.949459    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-bound-sa-token\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:46.949577 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.949527    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lklrw\" (UniqueName: \"kubernetes.io/projected/03eb4687-6b09-4c00-8601-4f7e11f8de4c-kube-api-access-lklrw\") pod \"ingress-canary-25dj7\" (UID: \"03eb4687-6b09-4c00-8601-4f7e11f8de4c\") " pod="openshift-ingress-canary/ingress-canary-25dj7"
Jul 11 23:31:46.949654 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.949638    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvnrl\" (UniqueName: \"kubernetes.io/projected/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-kube-api-access-cvnrl\") pod \"dns-default-jvtd6\" (UID: \"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf\") " pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:31:46.950000 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:46.949986    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsrhl\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-kube-api-access-zsrhl\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:47.056292 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:47.056255    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-25dj7"
Jul 11 23:31:47.188648 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:47.188035    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-25dj7"]
Jul 11 23:31:47.191702 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:47.191674    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03eb4687_6b09_4c00_8601_4f7e11f8de4c.slice/crio-e9522b981d77a72c643063650a00849afe6bad13cd76b300692e27735281d3f4 WatchSource:0}: Error finding container e9522b981d77a72c643063650a00849afe6bad13cd76b300692e27735281d3f4: Status 404 returned error can't find the container with id e9522b981d77a72c643063650a00849afe6bad13cd76b300692e27735281d3f4
Jul 11 23:31:47.196274 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:47.196247    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-25dj7" event={"ID":"03eb4687-6b09-4c00-8601-4f7e11f8de4c","Type":"ContainerStarted","Data":"e9522b981d77a72c643063650a00849afe6bad13cd76b300692e27735281d3f4"}
Jul 11 23:31:47.198915 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:47.198891    2569 generic.go:356] "Generic (PLEG): container finished" podID="0835749c-5f54-425f-99de-161c4b66c81a" containerID="8183b65b9c8bf16189899e8c97fdf0cb069658a2b492b30c59dd43e9e726eeb4" exitCode=0
Jul 11 23:31:47.199004 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:47.198943    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jwbv4" event={"ID":"0835749c-5f54-425f-99de-161c4b66c81a","Type":"ContainerDied","Data":"8183b65b9c8bf16189899e8c97fdf0cb069658a2b492b30c59dd43e9e726eeb4"}
Jul 11 23:31:47.446096 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:47.445925    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-metrics-tls\") pod \"dns-default-jvtd6\" (UID: \"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf\") " pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:31:47.446254 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:47.446122    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:47.446254 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:47.446077    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 11 23:31:47.446254 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:47.446236    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-metrics-tls podName:b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf nodeName:}" failed. No retries permitted until 2026-07-11 23:31:48.446217559 +0000 UTC m=+38.024310316 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-metrics-tls") pod "dns-default-jvtd6" (UID: "b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf") : secret "dns-default-metrics-tls" not found
Jul 11 23:31:47.446417 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:47.446268    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 11 23:31:47.446417 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:47.446283    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f664cb986-z7cww: secret "image-registry-tls" not found
Jul 11 23:31:47.446417 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:47.446339    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls podName:3c1dc75e-e9b9-4f38-916c-51230cc19a38 nodeName:}" failed. No retries permitted until 2026-07-11 23:31:48.446321793 +0000 UTC m=+38.024414561 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls") pod "image-registry-7f664cb986-z7cww" (UID: "3c1dc75e-e9b9-4f38-916c-51230cc19a38") : secret "image-registry-tls" not found
Jul 11 23:31:47.977278 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:47.977240    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:31:47.977278 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:47.977279    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:31:47.977529 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:47.977245    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:31:47.980040 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:47.980018    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 11 23:31:47.980147 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:47.980018    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nxtg4\""
Jul 11 23:31:47.980147 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:47.980022    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 11 23:31:47.981557 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:47.981415    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qb5vv\""
Jul 11 23:31:47.981557 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:47.981442    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 11 23:31:47.981557 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:47.981446    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 11 23:31:48.205406 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:48.205371    2569 generic.go:356] "Generic (PLEG): container finished" podID="0835749c-5f54-425f-99de-161c4b66c81a" containerID="9bb8def3d0d8d241b3f55dc3f99f3e9877e7602759db0f8c99ae01e4488c5bb0" exitCode=0
Jul 11 23:31:48.205851 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:48.205459    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jwbv4" event={"ID":"0835749c-5f54-425f-99de-161c4b66c81a","Type":"ContainerDied","Data":"9bb8def3d0d8d241b3f55dc3f99f3e9877e7602759db0f8c99ae01e4488c5bb0"}
Jul 11 23:31:48.455343 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:48.455309    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:48.455580 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:48.455412    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-metrics-tls\") pod \"dns-default-jvtd6\" (UID: \"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf\") " pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:31:48.455580 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:48.455501    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 11 23:31:48.455580 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:48.455522    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f664cb986-z7cww: secret "image-registry-tls" not found
Jul 11 23:31:48.455580 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:48.455526    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 11 23:31:48.455731 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:48.455589    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-metrics-tls podName:b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf nodeName:}" failed. No retries permitted until 2026-07-11 23:31:50.45556855 +0000 UTC m=+40.033661308 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-metrics-tls") pod "dns-default-jvtd6" (UID: "b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf") : secret "dns-default-metrics-tls" not found
Jul 11 23:31:48.455731 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:48.455610    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls podName:3c1dc75e-e9b9-4f38-916c-51230cc19a38 nodeName:}" failed. No retries permitted until 2026-07-11 23:31:50.455600944 +0000 UTC m=+40.033693716 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls") pod "image-registry-7f664cb986-z7cww" (UID: "3c1dc75e-e9b9-4f38-916c-51230cc19a38") : secret "image-registry-tls" not found
Jul 11 23:31:49.212757 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:49.212707    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jwbv4" event={"ID":"0835749c-5f54-425f-99de-161c4b66c81a","Type":"ContainerStarted","Data":"7427fc51c99fe17ede0366cb016dee55f0554958c061ec6d0b5272e9621885ca"}
Jul 11 23:31:49.237749 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:49.237683    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jwbv4" podStartSLOduration=9.508721732 podStartE2EDuration="38.237663059s" totalImagesPullingTime="28.728941327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:49.23666769 +0000 UTC m=+38.814760514" watchObservedRunningTime="2026-07-11 23:31:49.237663059 +0000 UTC m=+38.815755840"
Jul 11 23:31:49.804328 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:49.804297    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6fqjd"]
Jul 11 23:31:49.822998 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:49.822971    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6fqjd"]
Jul 11 23:31:49.823129 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:49.823089    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6fqjd"
Jul 11 23:31:49.825681 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:49.825653    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-b2c7w\""
Jul 11 23:31:49.825681 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:49.825653    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 11 23:31:49.826467 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:49.826450    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 11 23:31:49.968196 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:49.968160    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvrvh\" (UniqueName: \"kubernetes.io/projected/7cd949dd-1c5d-4fb8-83c6-82adff61ed58-kube-api-access-kvrvh\") pod \"migrator-7f59b54f6c-6fqjd\" (UID: \"7cd949dd-1c5d-4fb8-83c6-82adff61ed58\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6fqjd"
Jul 11 23:31:50.068861 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.068819    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvrvh\" (UniqueName: \"kubernetes.io/projected/7cd949dd-1c5d-4fb8-83c6-82adff61ed58-kube-api-access-kvrvh\") pod \"migrator-7f59b54f6c-6fqjd\" (UID: \"7cd949dd-1c5d-4fb8-83c6-82adff61ed58\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6fqjd"
Jul 11 23:31:50.077934 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.077907    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvrvh\" (UniqueName: \"kubernetes.io/projected/7cd949dd-1c5d-4fb8-83c6-82adff61ed58-kube-api-access-kvrvh\") pod \"migrator-7f59b54f6c-6fqjd\" (UID: \"7cd949dd-1c5d-4fb8-83c6-82adff61ed58\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6fqjd"
Jul 11 23:31:50.131291 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.131259    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6fqjd"
Jul 11 23:31:50.218397 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.218358    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-25dj7" event={"ID":"03eb4687-6b09-4c00-8601-4f7e11f8de4c","Type":"ContainerStarted","Data":"e651fe44f7ef7109d4fc2322cf2cacb88994eaddcc30ea4cb2f8e313e743874d"}
Jul 11 23:31:50.237846 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.237415    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-25dj7" podStartSLOduration=2.095301921 podStartE2EDuration="4.237397097s" totalImagesPullingTime="2.142095176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:31:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:50.236740547 +0000 UTC m=+39.814833324" watchObservedRunningTime="2026-07-11 23:31:50.237397097 +0000 UTC m=+39.815489873"
Jul 11 23:31:50.276308 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.276277    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ftcz2"]
Jul 11 23:31:50.286162 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:50.286137    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cd949dd_1c5d_4fb8_83c6_82adff61ed58.slice/crio-37ecd6d8a368a0c5bbbff6ac9b70632a831299f0fe5ca84af4202ac8c7f86799 WatchSource:0}: Error finding container 37ecd6d8a368a0c5bbbff6ac9b70632a831299f0fe5ca84af4202ac8c7f86799: Status 404 returned error can't find the container with id 37ecd6d8a368a0c5bbbff6ac9b70632a831299f0fe5ca84af4202ac8c7f86799
Jul 11 23:31:50.298703 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.298676    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6fqjd"]
Jul 11 23:31:50.298796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.298727    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ftcz2"]
Jul 11 23:31:50.298796 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.298750    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:50.301252 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.301231    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 11 23:31:50.301348 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.301231    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 11 23:31:50.301348 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.301240    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k95xv\""
Jul 11 23:31:50.301633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.301613    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 11 23:31:50.301713 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.301619    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 11 23:31:50.472179 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.472074    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82d38c08-74a5-42db-b8bb-c45a7b78417e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:50.472179 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.472129    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/82d38c08-74a5-42db-b8bb-c45a7b78417e-data-volume\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:50.472179 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.472155    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-metrics-tls\") pod \"dns-default-jvtd6\" (UID: \"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf\") " pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:31:50.472382 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.472188    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82d38c08-74a5-42db-b8bb-c45a7b78417e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:50.472382 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.472209    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9g6k\" (UniqueName: \"kubernetes.io/projected/82d38c08-74a5-42db-b8bb-c45a7b78417e-kube-api-access-j9g6k\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:50.472382 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.472231    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:50.472382 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.472251    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/82d38c08-74a5-42db-b8bb-c45a7b78417e-crio-socket\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:50.472382 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:50.472313    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 11 23:31:50.472382 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:50.472366    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 11 23:31:50.472382 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:50.472380    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f664cb986-z7cww: secret "image-registry-tls" not found
Jul 11 23:31:50.472605 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:50.472452    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls podName:3c1dc75e-e9b9-4f38-916c-51230cc19a38 nodeName:}" failed. No retries permitted until 2026-07-11 23:31:54.472418532 +0000 UTC m=+44.050511303 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls") pod "image-registry-7f664cb986-z7cww" (UID: "3c1dc75e-e9b9-4f38-916c-51230cc19a38") : secret "image-registry-tls" not found
Jul 11 23:31:50.472605 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:50.472471    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-metrics-tls podName:b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf nodeName:}" failed. No retries permitted until 2026-07-11 23:31:54.472465269 +0000 UTC m=+44.050558023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-metrics-tls") pod "dns-default-jvtd6" (UID: "b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf") : secret "dns-default-metrics-tls" not found
Jul 11 23:31:50.573624 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.573589    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82d38c08-74a5-42db-b8bb-c45a7b78417e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:50.573779 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.573640    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/82d38c08-74a5-42db-b8bb-c45a7b78417e-data-volume\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:50.573779 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:50.573759    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 11 23:31:50.573851 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.573764    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82d38c08-74a5-42db-b8bb-c45a7b78417e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:50.573851 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:50.573826    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82d38c08-74a5-42db-b8bb-c45a7b78417e-insights-runtime-extractor-tls podName:82d38c08-74a5-42db-b8bb-c45a7b78417e nodeName:}" failed. No retries permitted until 2026-07-11 23:31:51.0738071 +0000 UTC m=+40.651899865 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82d38c08-74a5-42db-b8bb-c45a7b78417e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftcz2" (UID: "82d38c08-74a5-42db-b8bb-c45a7b78417e") : secret "insights-runtime-extractor-tls" not found
Jul 11 23:31:50.573851 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:50.573837    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 11 23:31:50.573943 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.573858    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9g6k\" (UniqueName: \"kubernetes.io/projected/82d38c08-74a5-42db-b8bb-c45a7b78417e-kube-api-access-j9g6k\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:50.573943 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:50.573885    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82d38c08-74a5-42db-b8bb-c45a7b78417e-kube-rbac-proxy-cm podName:82d38c08-74a5-42db-b8bb-c45a7b78417e nodeName:}" failed. No retries permitted until 2026-07-11 23:31:51.073872633 +0000 UTC m=+40.651965400 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/82d38c08-74a5-42db-b8bb-c45a7b78417e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ftcz2" (UID: "82d38c08-74a5-42db-b8bb-c45a7b78417e") : configmap "kube-rbac-proxy" not found
Jul 11 23:31:50.573943 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.573933    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/82d38c08-74a5-42db-b8bb-c45a7b78417e-crio-socket\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:50.574054 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.573969    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/82d38c08-74a5-42db-b8bb-c45a7b78417e-data-volume\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:50.574146 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.574132    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/82d38c08-74a5-42db-b8bb-c45a7b78417e-crio-socket\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:50.584567 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:50.584540    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9g6k\" (UniqueName: \"kubernetes.io/projected/82d38c08-74a5-42db-b8bb-c45a7b78417e-kube-api-access-j9g6k\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:51.078665 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:51.078631    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82d38c08-74a5-42db-b8bb-c45a7b78417e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:51.078881 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:51.078742    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82d38c08-74a5-42db-b8bb-c45a7b78417e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:51.078881 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:51.078765    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 11 23:31:51.078881 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:51.078844    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82d38c08-74a5-42db-b8bb-c45a7b78417e-kube-rbac-proxy-cm podName:82d38c08-74a5-42db-b8bb-c45a7b78417e nodeName:}" failed. No retries permitted until 2026-07-11 23:31:52.078825074 +0000 UTC m=+41.656917829 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/82d38c08-74a5-42db-b8bb-c45a7b78417e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ftcz2" (UID: "82d38c08-74a5-42db-b8bb-c45a7b78417e") : configmap "kube-rbac-proxy" not found
Jul 11 23:31:51.078881 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:51.078858    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 11 23:31:51.079081 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:51.078922    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82d38c08-74a5-42db-b8bb-c45a7b78417e-insights-runtime-extractor-tls podName:82d38c08-74a5-42db-b8bb-c45a7b78417e nodeName:}" failed. No retries permitted until 2026-07-11 23:31:52.078903659 +0000 UTC m=+41.656996414 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82d38c08-74a5-42db-b8bb-c45a7b78417e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftcz2" (UID: "82d38c08-74a5-42db-b8bb-c45a7b78417e") : secret "insights-runtime-extractor-tls" not found
Jul 11 23:31:51.221547 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:51.221508    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6fqjd" event={"ID":"7cd949dd-1c5d-4fb8-83c6-82adff61ed58","Type":"ContainerStarted","Data":"37ecd6d8a368a0c5bbbff6ac9b70632a831299f0fe5ca84af4202ac8c7f86799"}
Jul 11 23:31:52.087720 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.087695    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82d38c08-74a5-42db-b8bb-c45a7b78417e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:52.087840 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.087764    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82d38c08-74a5-42db-b8bb-c45a7b78417e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:52.087840 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:52.087829    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 11 23:31:52.087942 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:52.087865    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 11 23:31:52.087942 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:52.087909    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82d38c08-74a5-42db-b8bb-c45a7b78417e-kube-rbac-proxy-cm podName:82d38c08-74a5-42db-b8bb-c45a7b78417e nodeName:}" failed. No retries permitted until 2026-07-11 23:31:54.087888653 +0000 UTC m=+43.665981421 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/82d38c08-74a5-42db-b8bb-c45a7b78417e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ftcz2" (UID: "82d38c08-74a5-42db-b8bb-c45a7b78417e") : configmap "kube-rbac-proxy" not found
Jul 11 23:31:52.087942 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:52.087931    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82d38c08-74a5-42db-b8bb-c45a7b78417e-insights-runtime-extractor-tls podName:82d38c08-74a5-42db-b8bb-c45a7b78417e nodeName:}" failed. No retries permitted until 2026-07-11 23:31:54.087920485 +0000 UTC m=+43.666013263 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82d38c08-74a5-42db-b8bb-c45a7b78417e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftcz2" (UID: "82d38c08-74a5-42db-b8bb-c45a7b78417e") : secret "insights-runtime-extractor-tls" not found
Jul 11 23:31:52.225970 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.225897    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6fqjd" event={"ID":"7cd949dd-1c5d-4fb8-83c6-82adff61ed58","Type":"ContainerStarted","Data":"5b972dfabe0da2bfd4ec8380ccb40974ae61756e402622dd38e30c5a9192bed7"}
Jul 11 23:31:52.225970 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.225950    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6fqjd" event={"ID":"7cd949dd-1c5d-4fb8-83c6-82adff61ed58","Type":"ContainerStarted","Data":"6e666e0f2e2b040bfe7bcb6edeb572661dfb4c510f9ba8897e2ad2293f843943"}
Jul 11 23:31:52.240527 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.240478    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-6fqjd" podStartSLOduration=1.6080741920000001 podStartE2EDuration="3.240465144s" totalImagesPullingTime="1.632390952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:31:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:52.239657138 +0000 UTC m=+41.817749908" watchObservedRunningTime="2026-07-11 23:31:52.240465144 +0000 UTC m=+41.818557920"
Jul 11 23:31:52.438615 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.438581    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-ts7bj"]
Jul 11 23:31:52.458913 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.458885    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-ts7bj"]
Jul 11 23:31:52.459046 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.458982    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-ts7bj"
Jul 11 23:31:52.461610 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.461587    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 11 23:31:52.461750 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.461663    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 11 23:31:52.461750 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.461697    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-8knmc\""
Jul 11 23:31:52.461750 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.461698    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 11 23:31:52.461880 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.461798    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 11 23:31:52.592447 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.592396    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/766da8a7-0d55-4b52-9cf2-843ebfdc24e8-signing-key\") pod \"service-ca-79f68b9f4f-ts7bj\" (UID: \"766da8a7-0d55-4b52-9cf2-843ebfdc24e8\") " pod="openshift-service-ca/service-ca-79f68b9f4f-ts7bj"
Jul 11 23:31:52.592613 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.592544    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/766da8a7-0d55-4b52-9cf2-843ebfdc24e8-signing-cabundle\") pod \"service-ca-79f68b9f4f-ts7bj\" (UID: \"766da8a7-0d55-4b52-9cf2-843ebfdc24e8\") " pod="openshift-service-ca/service-ca-79f68b9f4f-ts7bj"
Jul 11 23:31:52.592613 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.592571    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzp7q\" (UniqueName: \"kubernetes.io/projected/766da8a7-0d55-4b52-9cf2-843ebfdc24e8-kube-api-access-zzp7q\") pod \"service-ca-79f68b9f4f-ts7bj\" (UID: \"766da8a7-0d55-4b52-9cf2-843ebfdc24e8\") " pod="openshift-service-ca/service-ca-79f68b9f4f-ts7bj"
Jul 11 23:31:52.693738 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.693706    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/766da8a7-0d55-4b52-9cf2-843ebfdc24e8-signing-cabundle\") pod \"service-ca-79f68b9f4f-ts7bj\" (UID: \"766da8a7-0d55-4b52-9cf2-843ebfdc24e8\") " pod="openshift-service-ca/service-ca-79f68b9f4f-ts7bj"
Jul 11 23:31:52.693738 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.693739    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzp7q\" (UniqueName: \"kubernetes.io/projected/766da8a7-0d55-4b52-9cf2-843ebfdc24e8-kube-api-access-zzp7q\") pod \"service-ca-79f68b9f4f-ts7bj\" (UID: \"766da8a7-0d55-4b52-9cf2-843ebfdc24e8\") " pod="openshift-service-ca/service-ca-79f68b9f4f-ts7bj"
Jul 11 23:31:52.693910 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.693775    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/766da8a7-0d55-4b52-9cf2-843ebfdc24e8-signing-key\") pod \"service-ca-79f68b9f4f-ts7bj\" (UID: \"766da8a7-0d55-4b52-9cf2-843ebfdc24e8\") " pod="openshift-service-ca/service-ca-79f68b9f4f-ts7bj"
Jul 11 23:31:52.694410 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.694383    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/766da8a7-0d55-4b52-9cf2-843ebfdc24e8-signing-cabundle\") pod \"service-ca-79f68b9f4f-ts7bj\" (UID: \"766da8a7-0d55-4b52-9cf2-843ebfdc24e8\") " pod="openshift-service-ca/service-ca-79f68b9f4f-ts7bj"
Jul 11 23:31:52.696540 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.696522    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/766da8a7-0d55-4b52-9cf2-843ebfdc24e8-signing-key\") pod \"service-ca-79f68b9f4f-ts7bj\" (UID: \"766da8a7-0d55-4b52-9cf2-843ebfdc24e8\") " pod="openshift-service-ca/service-ca-79f68b9f4f-ts7bj"
Jul 11 23:31:52.718996 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.718973    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzp7q\" (UniqueName: \"kubernetes.io/projected/766da8a7-0d55-4b52-9cf2-843ebfdc24e8-kube-api-access-zzp7q\") pod \"service-ca-79f68b9f4f-ts7bj\" (UID: \"766da8a7-0d55-4b52-9cf2-843ebfdc24e8\") " pod="openshift-service-ca/service-ca-79f68b9f4f-ts7bj"
Jul 11 23:31:52.767964 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.767932    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-ts7bj"
Jul 11 23:31:52.868571 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.868491    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-smg2m_fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7/dns-node-resolver/0.log"
Jul 11 23:31:52.920823 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:52.920789    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-ts7bj"]
Jul 11 23:31:52.924845 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:31:52.924818    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod766da8a7_0d55_4b52_9cf2_843ebfdc24e8.slice/crio-f8d01647ab0da76a49b355044ca35b7283ccab3f8172b84de442353e949307c7 WatchSource:0}: Error finding container f8d01647ab0da76a49b355044ca35b7283ccab3f8172b84de442353e949307c7: Status 404 returned error can't find the container with id f8d01647ab0da76a49b355044ca35b7283ccab3f8172b84de442353e949307c7
Jul 11 23:31:53.232664 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:53.232573    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-ts7bj" event={"ID":"766da8a7-0d55-4b52-9cf2-843ebfdc24e8","Type":"ContainerStarted","Data":"f8d01647ab0da76a49b355044ca35b7283ccab3f8172b84de442353e949307c7"}
Jul 11 23:31:53.868679 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:53.868654    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mlkk5_ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790/node-ca/0.log"
Jul 11 23:31:54.106626 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:54.106589    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82d38c08-74a5-42db-b8bb-c45a7b78417e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:54.106869 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:54.106678    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82d38c08-74a5-42db-b8bb-c45a7b78417e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:54.106869 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:54.106769    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 11 23:31:54.106869 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:54.106815    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 11 23:31:54.106869 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:54.106838    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82d38c08-74a5-42db-b8bb-c45a7b78417e-insights-runtime-extractor-tls podName:82d38c08-74a5-42db-b8bb-c45a7b78417e nodeName:}" failed. No retries permitted until 2026-07-11 23:31:58.106821555 +0000 UTC m=+47.684914325 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82d38c08-74a5-42db-b8bb-c45a7b78417e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftcz2" (UID: "82d38c08-74a5-42db-b8bb-c45a7b78417e") : secret "insights-runtime-extractor-tls" not found
Jul 11 23:31:54.107059 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:54.106896    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82d38c08-74a5-42db-b8bb-c45a7b78417e-kube-rbac-proxy-cm podName:82d38c08-74a5-42db-b8bb-c45a7b78417e nodeName:}" failed. No retries permitted until 2026-07-11 23:31:58.106876693 +0000 UTC m=+47.684969461 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/82d38c08-74a5-42db-b8bb-c45a7b78417e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ftcz2" (UID: "82d38c08-74a5-42db-b8bb-c45a7b78417e") : configmap "kube-rbac-proxy" not found
Jul 11 23:31:54.268968 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:54.268898    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-25dj7_03eb4687-6b09-4c00-8601-4f7e11f8de4c/serve-healthcheck-canary/0.log"
Jul 11 23:31:54.510070 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:54.510030    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-metrics-tls\") pod \"dns-default-jvtd6\" (UID: \"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf\") " pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:31:54.510234 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:54.510090    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:31:54.510234 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:54.510201    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 11 23:31:54.510234 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:54.510205    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 11 23:31:54.510397 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:54.510274    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-metrics-tls podName:b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf nodeName:}" failed. No retries permitted until 2026-07-11 23:32:02.510259217 +0000 UTC m=+52.088351971 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-metrics-tls") pod "dns-default-jvtd6" (UID: "b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf") : secret "dns-default-metrics-tls" not found
Jul 11 23:31:54.510397 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:54.510214    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f664cb986-z7cww: secret "image-registry-tls" not found
Jul 11 23:31:54.510397 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:54.510346    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls podName:3c1dc75e-e9b9-4f38-916c-51230cc19a38 nodeName:}" failed. No retries permitted until 2026-07-11 23:32:02.510333993 +0000 UTC m=+52.088426746 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls") pod "image-registry-7f664cb986-z7cww" (UID: "3c1dc75e-e9b9-4f38-916c-51230cc19a38") : secret "image-registry-tls" not found
Jul 11 23:31:54.668748 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:54.668690    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-6fqjd_7cd949dd-1c5d-4fb8-83c6-82adff61ed58/migrator/0.log"
Jul 11 23:31:54.880247 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:54.880218    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-6fqjd_7cd949dd-1c5d-4fb8-83c6-82adff61ed58/graceful-termination/0.log"
Jul 11 23:31:56.242072 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:56.242037    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-ts7bj" event={"ID":"766da8a7-0d55-4b52-9cf2-843ebfdc24e8","Type":"ContainerStarted","Data":"773dcff5f1b9a21fa041bc0576b89a49b8c14172037f6002e1ceb1e3f7bc1859"}
Jul 11 23:31:56.258945 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:56.258870    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-ts7bj" podStartSLOduration=1.8818799400000001 podStartE2EDuration="4.258856919s" totalImagesPullingTime="2.376976979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:31:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:56.258811887 +0000 UTC m=+45.836904663" watchObservedRunningTime="2026-07-11 23:31:56.258856919 +0000 UTC m=+45.836949695"
Jul 11 23:31:58.140074 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:58.140031    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82d38c08-74a5-42db-b8bb-c45a7b78417e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:58.140540 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:58.140101    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82d38c08-74a5-42db-b8bb-c45a7b78417e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:31:58.140540 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:58.140192    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 11 23:31:58.140540 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:31:58.140273    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82d38c08-74a5-42db-b8bb-c45a7b78417e-insights-runtime-extractor-tls podName:82d38c08-74a5-42db-b8bb-c45a7b78417e nodeName:}" failed. No retries permitted until 2026-07-11 23:32:06.140252972 +0000 UTC m=+55.718345742 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82d38c08-74a5-42db-b8bb-c45a7b78417e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ftcz2" (UID: "82d38c08-74a5-42db-b8bb-c45a7b78417e") : secret "insights-runtime-extractor-tls" not found
Jul 11 23:31:58.140701 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:31:58.140559    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82d38c08-74a5-42db-b8bb-c45a7b78417e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:32:02.576984 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:02.576934    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-metrics-tls\") pod \"dns-default-jvtd6\" (UID: \"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf\") " pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:32:02.576984 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:02.576985    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:32:02.579257 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:02.579222    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf-metrics-tls\") pod \"dns-default-jvtd6\" (UID: \"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf\") " pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:32:02.579381 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:02.579331    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls\") pod \"image-registry-7f664cb986-z7cww\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") " pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:32:02.639006 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:02.638972    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:32:02.664453 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:02.664410    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:32:02.801569 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:02.801504    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f664cb986-z7cww"]
Jul 11 23:32:02.805202 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:32:02.805171    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c1dc75e_e9b9_4f38_916c_51230cc19a38.slice/crio-149e2091a759e912a5d4923170255f5ca104a486a648395e0731101dfc656bff WatchSource:0}: Error finding container 149e2091a759e912a5d4923170255f5ca104a486a648395e0731101dfc656bff: Status 404 returned error can't find the container with id 149e2091a759e912a5d4923170255f5ca104a486a648395e0731101dfc656bff
Jul 11 23:32:02.819492 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:02.819462    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jvtd6"]
Jul 11 23:32:02.823090 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:32:02.823065    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3ea3f00_d48e_4ea8_8a9d_50c49ff4c9bf.slice/crio-f62ec0cf5e4d0fdd3398a6483dd9f44c278fa92bcc2e414f4ab61fa439539260 WatchSource:0}: Error finding container f62ec0cf5e4d0fdd3398a6483dd9f44c278fa92bcc2e414f4ab61fa439539260: Status 404 returned error can't find the container with id f62ec0cf5e4d0fdd3398a6483dd9f44c278fa92bcc2e414f4ab61fa439539260
Jul 11 23:32:03.265769 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:03.265679    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jvtd6" event={"ID":"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf","Type":"ContainerStarted","Data":"f62ec0cf5e4d0fdd3398a6483dd9f44c278fa92bcc2e414f4ab61fa439539260"}
Jul 11 23:32:03.267122 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:03.267098    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f664cb986-z7cww" event={"ID":"3c1dc75e-e9b9-4f38-916c-51230cc19a38","Type":"ContainerStarted","Data":"4a89b6f2aaf6c867727b9ff78243fd34e582a4d5bda39153ea40caa0808ef8a5"}
Jul 11 23:32:03.267243 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:03.267143    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f664cb986-z7cww" event={"ID":"3c1dc75e-e9b9-4f38-916c-51230cc19a38","Type":"ContainerStarted","Data":"149e2091a759e912a5d4923170255f5ca104a486a648395e0731101dfc656bff"}
Jul 11 23:32:03.267243 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:03.267238    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:32:03.285991 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:03.285927    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7f664cb986-z7cww" podStartSLOduration=26.285906853 podStartE2EDuration="26.285906853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:31:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:03.284778827 +0000 UTC m=+52.862871603" watchObservedRunningTime="2026-07-11 23:32:03.285906853 +0000 UTC m=+52.863999630"
Jul 11 23:32:05.275637 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:05.275547    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jvtd6" event={"ID":"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf","Type":"ContainerStarted","Data":"76950d8d821d03670efa1078028171be0c414e3201af827478a65246723d27dd"}
Jul 11 23:32:05.275637 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:05.275611    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jvtd6" event={"ID":"b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf","Type":"ContainerStarted","Data":"548bdcd1c8bc472836146e97875dc698ee342d157925c84c367cd2a5a4730ddb"}
Jul 11 23:32:05.276105 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:05.275676    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:32:05.291646 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:05.291598    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jvtd6" podStartSLOduration=17.226494446 podStartE2EDuration="19.291582407s" totalImagesPullingTime="2.065087961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:31:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:05.291065135 +0000 UTC m=+54.869157911" watchObservedRunningTime="2026-07-11 23:32:05.291582407 +0000 UTC m=+54.869675185"
Jul 11 23:32:06.207417 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:06.207381    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82d38c08-74a5-42db-b8bb-c45a7b78417e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:32:06.209670 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:06.209647    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82d38c08-74a5-42db-b8bb-c45a7b78417e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ftcz2\" (UID: \"82d38c08-74a5-42db-b8bb-c45a7b78417e\") " pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:32:06.507836 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:06.507756    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ftcz2"
Jul 11 23:32:06.632243 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:06.632202    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ftcz2"]
Jul 11 23:32:06.636321 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:32:06.636296    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82d38c08_74a5_42db_b8bb_c45a7b78417e.slice/crio-8bea7dfd52a9d2a8205b66dd0dee3220ca944c935b199dab02e69d021f5c97ca WatchSource:0}: Error finding container 8bea7dfd52a9d2a8205b66dd0dee3220ca944c935b199dab02e69d021f5c97ca: Status 404 returned error can't find the container with id 8bea7dfd52a9d2a8205b66dd0dee3220ca944c935b199dab02e69d021f5c97ca
Jul 11 23:32:07.284663 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:07.284622    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ftcz2" event={"ID":"82d38c08-74a5-42db-b8bb-c45a7b78417e","Type":"ContainerStarted","Data":"8bea7dfd52a9d2a8205b66dd0dee3220ca944c935b199dab02e69d021f5c97ca"}
Jul 11 23:32:11.298379 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:11.298334    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ftcz2" event={"ID":"82d38c08-74a5-42db-b8bb-c45a7b78417e","Type":"ContainerStarted","Data":"eb83da3ed94631cc86abf0c2b1675d722d6df4de1aa819c0235eb0ea1960d6dc"}
Jul 11 23:32:12.304052 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:12.304008    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ftcz2" event={"ID":"82d38c08-74a5-42db-b8bb-c45a7b78417e","Type":"ContainerStarted","Data":"c59d86cefbb8f7725ae307e1a8b9dd766d724aa89a82da202205e3c6eaad9466"}
Jul 11 23:32:13.194533 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:13.194496    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xd2gc"
Jul 11 23:32:14.312480 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.312450    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ftcz2" event={"ID":"82d38c08-74a5-42db-b8bb-c45a7b78417e","Type":"ContainerStarted","Data":"f27d377e050afbaf6e649822f6b3c1a50cdfa4687be78133eb04a7d3b86296a7"}
Jul 11 23:32:14.328203 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.328148    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ftcz2" podStartSLOduration=17.317354504 podStartE2EDuration="24.32812948s" totalImagesPullingTime="7.010774976s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:14.32789498 +0000 UTC m=+63.905987767" watchObservedRunningTime="2026-07-11 23:32:14.32812948 +0000 UTC m=+63.906222255"
Jul 11 23:32:14.706887 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.706815    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7"]
Jul 11 23:32:14.709176 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.709115    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7"
Jul 11 23:32:14.710131 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.710113    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl"]
Jul 11 23:32:14.714973 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.712939    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"]
Jul 11 23:32:14.715463 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.715420    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl"
Jul 11 23:32:14.715783 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.715666    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 11 23:32:14.715783 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.715673    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 11 23:32:14.715783 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.715673    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 11 23:32:14.715968 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.715825    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 11 23:32:14.716370 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.716356    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.718134 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.718112    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 11 23:32:14.718233 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.718116    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-qnhlm\""
Jul 11 23:32:14.719035 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.718835    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 11 23:32:14.719035 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.718847    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 11 23:32:14.719035 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.718898    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 11 23:32:14.719262 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.719190    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 11 23:32:14.720001 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.719981    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7"]
Jul 11 23:32:14.723082 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.723063    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl"]
Jul 11 23:32:14.740019 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.739999    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"]
Jul 11 23:32:14.781049 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.781021    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5c5435b8-e1a0-4621-8c12-65c94bd5f8c1-klusterlet-config\") pod \"klusterlet-addon-workmgr-7ff448f655-8ktw7\" (UID: \"5c5435b8-e1a0-4621-8c12-65c94bd5f8c1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7"
Jul 11 23:32:14.781240 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.781056    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/54db0b92-a110-4b17-ae95-a58aa3fed9e0-ca\") pod \"cluster-proxy-proxy-agent-64768c7f89-nw27t\" (UID: \"54db0b92-a110-4b17-ae95-a58aa3fed9e0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.781240 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.781090    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf8vq\" (UniqueName: \"kubernetes.io/projected/5c5435b8-e1a0-4621-8c12-65c94bd5f8c1-kube-api-access-zf8vq\") pod \"klusterlet-addon-workmgr-7ff448f655-8ktw7\" (UID: \"5c5435b8-e1a0-4621-8c12-65c94bd5f8c1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7"
Jul 11 23:32:14.781240 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.781129    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/54db0b92-a110-4b17-ae95-a58aa3fed9e0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-64768c7f89-nw27t\" (UID: \"54db0b92-a110-4b17-ae95-a58aa3fed9e0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.781240 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.781170    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qsgd\" (UniqueName: \"kubernetes.io/projected/0a6cb4fa-09c9-4ebd-8cc6-041e8a65d86a-kube-api-access-2qsgd\") pod \"managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl\" (UID: \"0a6cb4fa-09c9-4ebd-8cc6-041e8a65d86a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl"
Jul 11 23:32:14.781240 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.781204    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/54db0b92-a110-4b17-ae95-a58aa3fed9e0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-64768c7f89-nw27t\" (UID: \"54db0b92-a110-4b17-ae95-a58aa3fed9e0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.781240 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.781232    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0a6cb4fa-09c9-4ebd-8cc6-041e8a65d86a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl\" (UID: \"0a6cb4fa-09c9-4ebd-8cc6-041e8a65d86a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl"
Jul 11 23:32:14.781512 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.781255    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gfkw\" (UniqueName: \"kubernetes.io/projected/54db0b92-a110-4b17-ae95-a58aa3fed9e0-kube-api-access-4gfkw\") pod \"cluster-proxy-proxy-agent-64768c7f89-nw27t\" (UID: \"54db0b92-a110-4b17-ae95-a58aa3fed9e0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.781512 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.781276    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c5435b8-e1a0-4621-8c12-65c94bd5f8c1-tmp\") pod \"klusterlet-addon-workmgr-7ff448f655-8ktw7\" (UID: \"5c5435b8-e1a0-4621-8c12-65c94bd5f8c1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7"
Jul 11 23:32:14.781512 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.781324    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/54db0b92-a110-4b17-ae95-a58aa3fed9e0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-64768c7f89-nw27t\" (UID: \"54db0b92-a110-4b17-ae95-a58aa3fed9e0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.781512 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.781376    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/54db0b92-a110-4b17-ae95-a58aa3fed9e0-hub\") pod \"cluster-proxy-proxy-agent-64768c7f89-nw27t\" (UID: \"54db0b92-a110-4b17-ae95-a58aa3fed9e0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.815697 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.815661    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f664cb986-z7cww"]
Jul 11 23:32:14.882215 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.882185    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf8vq\" (UniqueName: \"kubernetes.io/projected/5c5435b8-e1a0-4621-8c12-65c94bd5f8c1-kube-api-access-zf8vq\") pod \"klusterlet-addon-workmgr-7ff448f655-8ktw7\" (UID: \"5c5435b8-e1a0-4621-8c12-65c94bd5f8c1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7"
Jul 11 23:32:14.882410 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.882227    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/54db0b92-a110-4b17-ae95-a58aa3fed9e0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-64768c7f89-nw27t\" (UID: \"54db0b92-a110-4b17-ae95-a58aa3fed9e0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.882410 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.882274    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qsgd\" (UniqueName: \"kubernetes.io/projected/0a6cb4fa-09c9-4ebd-8cc6-041e8a65d86a-kube-api-access-2qsgd\") pod \"managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl\" (UID: \"0a6cb4fa-09c9-4ebd-8cc6-041e8a65d86a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl"
Jul 11 23:32:14.882410 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.882306    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/54db0b92-a110-4b17-ae95-a58aa3fed9e0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-64768c7f89-nw27t\" (UID: \"54db0b92-a110-4b17-ae95-a58aa3fed9e0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.882410 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.882337    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0a6cb4fa-09c9-4ebd-8cc6-041e8a65d86a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl\" (UID: \"0a6cb4fa-09c9-4ebd-8cc6-041e8a65d86a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl"
Jul 11 23:32:14.882410 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.882358    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gfkw\" (UniqueName: \"kubernetes.io/projected/54db0b92-a110-4b17-ae95-a58aa3fed9e0-kube-api-access-4gfkw\") pod \"cluster-proxy-proxy-agent-64768c7f89-nw27t\" (UID: \"54db0b92-a110-4b17-ae95-a58aa3fed9e0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.882660 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.882489    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c5435b8-e1a0-4621-8c12-65c94bd5f8c1-tmp\") pod \"klusterlet-addon-workmgr-7ff448f655-8ktw7\" (UID: \"5c5435b8-e1a0-4621-8c12-65c94bd5f8c1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7"
Jul 11 23:32:14.882660 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.882582    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/54db0b92-a110-4b17-ae95-a58aa3fed9e0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-64768c7f89-nw27t\" (UID: \"54db0b92-a110-4b17-ae95-a58aa3fed9e0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.882660 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.882612    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/54db0b92-a110-4b17-ae95-a58aa3fed9e0-hub\") pod \"cluster-proxy-proxy-agent-64768c7f89-nw27t\" (UID: \"54db0b92-a110-4b17-ae95-a58aa3fed9e0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.882781 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.882675    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5c5435b8-e1a0-4621-8c12-65c94bd5f8c1-klusterlet-config\") pod \"klusterlet-addon-workmgr-7ff448f655-8ktw7\" (UID: \"5c5435b8-e1a0-4621-8c12-65c94bd5f8c1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7"
Jul 11 23:32:14.882781 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.882704    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/54db0b92-a110-4b17-ae95-a58aa3fed9e0-ca\") pod \"cluster-proxy-proxy-agent-64768c7f89-nw27t\" (UID: \"54db0b92-a110-4b17-ae95-a58aa3fed9e0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.882931 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.882910    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c5435b8-e1a0-4621-8c12-65c94bd5f8c1-tmp\") pod \"klusterlet-addon-workmgr-7ff448f655-8ktw7\" (UID: \"5c5435b8-e1a0-4621-8c12-65c94bd5f8c1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7"
Jul 11 23:32:14.883388 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.883364    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/54db0b92-a110-4b17-ae95-a58aa3fed9e0-additional-service-ca\") pod \"cluster-proxy-proxy-agent-64768c7f89-nw27t\" (UID: \"54db0b92-a110-4b17-ae95-a58aa3fed9e0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.885328 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.885289    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0a6cb4fa-09c9-4ebd-8cc6-041e8a65d86a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl\" (UID: \"0a6cb4fa-09c9-4ebd-8cc6-041e8a65d86a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl"
Jul 11 23:32:14.885328 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.885294    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/54db0b92-a110-4b17-ae95-a58aa3fed9e0-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-64768c7f89-nw27t\" (UID: \"54db0b92-a110-4b17-ae95-a58aa3fed9e0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.885508 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.885344    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/54db0b92-a110-4b17-ae95-a58aa3fed9e0-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-64768c7f89-nw27t\" (UID: \"54db0b92-a110-4b17-ae95-a58aa3fed9e0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.885508 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.885365    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/5c5435b8-e1a0-4621-8c12-65c94bd5f8c1-klusterlet-config\") pod \"klusterlet-addon-workmgr-7ff448f655-8ktw7\" (UID: \"5c5435b8-e1a0-4621-8c12-65c94bd5f8c1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7"
Jul 11 23:32:14.885585 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.885528    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/54db0b92-a110-4b17-ae95-a58aa3fed9e0-ca\") pod \"cluster-proxy-proxy-agent-64768c7f89-nw27t\" (UID: \"54db0b92-a110-4b17-ae95-a58aa3fed9e0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.885628 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.885592    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/54db0b92-a110-4b17-ae95-a58aa3fed9e0-hub\") pod \"cluster-proxy-proxy-agent-64768c7f89-nw27t\" (UID: \"54db0b92-a110-4b17-ae95-a58aa3fed9e0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.890415 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.890397    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf8vq\" (UniqueName: \"kubernetes.io/projected/5c5435b8-e1a0-4621-8c12-65c94bd5f8c1-kube-api-access-zf8vq\") pod \"klusterlet-addon-workmgr-7ff448f655-8ktw7\" (UID: \"5c5435b8-e1a0-4621-8c12-65c94bd5f8c1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7"
Jul 11 23:32:14.890794 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.890776    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gfkw\" (UniqueName: \"kubernetes.io/projected/54db0b92-a110-4b17-ae95-a58aa3fed9e0-kube-api-access-4gfkw\") pod \"cluster-proxy-proxy-agent-64768c7f89-nw27t\" (UID: \"54db0b92-a110-4b17-ae95-a58aa3fed9e0\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:14.890924 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.890902    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qsgd\" (UniqueName: \"kubernetes.io/projected/0a6cb4fa-09c9-4ebd-8cc6-041e8a65d86a-kube-api-access-2qsgd\") pod \"managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl\" (UID: \"0a6cb4fa-09c9-4ebd-8cc6-041e8a65d86a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl"
Jul 11 23:32:14.912347 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.912319    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7b7c4b75dd-lgsth"]
Jul 11 23:32:14.914994 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.914978    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:14.926030 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.926008    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b7c4b75dd-lgsth"]
Jul 11 23:32:14.983470 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.983366    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ee947d7e-27bc-4c75-b148-a2f95295c313-trusted-ca\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:14.983470 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.983452    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ee947d7e-27bc-4c75-b148-a2f95295c313-installation-pull-secrets\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:14.983470 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.983477    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wr88\" (UniqueName: \"kubernetes.io/projected/ee947d7e-27bc-4c75-b148-a2f95295c313-kube-api-access-2wr88\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:14.983665 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.983498    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ee947d7e-27bc-4c75-b148-a2f95295c313-image-registry-private-configuration\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:14.983665 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.983582    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ee947d7e-27bc-4c75-b148-a2f95295c313-ca-trust-extracted\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:14.983665 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.983652    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee947d7e-27bc-4c75-b148-a2f95295c313-bound-sa-token\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:14.983754 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.983683    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ee947d7e-27bc-4c75-b148-a2f95295c313-registry-tls\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:14.983754 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:14.983722    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ee947d7e-27bc-4c75-b148-a2f95295c313-registry-certificates\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:15.024559 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.024529    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7"
Jul 11 23:32:15.043692 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.043655    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl"
Jul 11 23:32:15.050616 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.050584    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"
Jul 11 23:32:15.084703 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.084663    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ee947d7e-27bc-4c75-b148-a2f95295c313-installation-pull-secrets\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:15.084873 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.084768    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wr88\" (UniqueName: \"kubernetes.io/projected/ee947d7e-27bc-4c75-b148-a2f95295c313-kube-api-access-2wr88\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:15.084953 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.084879    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ee947d7e-27bc-4c75-b148-a2f95295c313-image-registry-private-configuration\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:15.085011 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.084955    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ee947d7e-27bc-4c75-b148-a2f95295c313-ca-trust-extracted\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:15.085068 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.085014    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee947d7e-27bc-4c75-b148-a2f95295c313-bound-sa-token\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:15.085126 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.085054    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ee947d7e-27bc-4c75-b148-a2f95295c313-registry-tls\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:15.085184 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.085136    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ee947d7e-27bc-4c75-b148-a2f95295c313-registry-certificates\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:15.085184 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.085167    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ee947d7e-27bc-4c75-b148-a2f95295c313-trusted-ca\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:15.086534 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.085756    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ee947d7e-27bc-4c75-b148-a2f95295c313-ca-trust-extracted\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:15.086534 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.086469    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ee947d7e-27bc-4c75-b148-a2f95295c313-registry-certificates\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:15.086765 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.086595    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ee947d7e-27bc-4c75-b148-a2f95295c313-trusted-ca\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:15.088487 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.087942    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ee947d7e-27bc-4c75-b148-a2f95295c313-installation-pull-secrets\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:15.088487 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.088379    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ee947d7e-27bc-4c75-b148-a2f95295c313-registry-tls\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:15.088672 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.088538    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ee947d7e-27bc-4c75-b148-a2f95295c313-image-registry-private-configuration\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:15.097638 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.097599    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee947d7e-27bc-4c75-b148-a2f95295c313-bound-sa-token\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:15.097797 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.097735    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wr88\" (UniqueName: \"kubernetes.io/projected/ee947d7e-27bc-4c75-b148-a2f95295c313-kube-api-access-2wr88\") pod \"image-registry-7b7c4b75dd-lgsth\" (UID: \"ee947d7e-27bc-4c75-b148-a2f95295c313\") " pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:15.172761 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.172719    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7"]
Jul 11 23:32:15.177754 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:32:15.177722    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c5435b8_e1a0_4621_8c12_65c94bd5f8c1.slice/crio-8c2b170297cf05154fd604c8e298d10cf16d42ac0a42742f3011788efae51a7f WatchSource:0}: Error finding container 8c2b170297cf05154fd604c8e298d10cf16d42ac0a42742f3011788efae51a7f: Status 404 returned error can't find the container with id 8c2b170297cf05154fd604c8e298d10cf16d42ac0a42742f3011788efae51a7f
Jul 11 23:32:15.192891 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.192852    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl"]
Jul 11 23:32:15.196034 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:32:15.196005    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a6cb4fa_09c9_4ebd_8cc6_041e8a65d86a.slice/crio-5d6c6e2f86e5c934e2e84adb8a6017ad1cc706fb52192599a0e96af834da0178 WatchSource:0}: Error finding container 5d6c6e2f86e5c934e2e84adb8a6017ad1cc706fb52192599a0e96af834da0178: Status 404 returned error can't find the container with id 5d6c6e2f86e5c934e2e84adb8a6017ad1cc706fb52192599a0e96af834da0178
Jul 11 23:32:15.208124 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.208098    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t"]
Jul 11 23:32:15.210867 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:32:15.210839    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod54db0b92_a110_4b17_ae95_a58aa3fed9e0.slice/crio-2bbc039113e3c509564be20f715b0bbc1ec24f31cbb07727805107e317bd781c WatchSource:0}: Error finding container 2bbc039113e3c509564be20f715b0bbc1ec24f31cbb07727805107e317bd781c: Status 404 returned error can't find the container with id 2bbc039113e3c509564be20f715b0bbc1ec24f31cbb07727805107e317bd781c
Jul 11 23:32:15.233416 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.233381    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:15.281202 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.281171    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jvtd6"
Jul 11 23:32:15.317339 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.317307    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl" event={"ID":"0a6cb4fa-09c9-4ebd-8cc6-041e8a65d86a","Type":"ContainerStarted","Data":"5d6c6e2f86e5c934e2e84adb8a6017ad1cc706fb52192599a0e96af834da0178"}
Jul 11 23:32:15.318628 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.318585    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7" event={"ID":"5c5435b8-e1a0-4621-8c12-65c94bd5f8c1","Type":"ContainerStarted","Data":"8c2b170297cf05154fd604c8e298d10cf16d42ac0a42742f3011788efae51a7f"}
Jul 11 23:32:15.319826 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.319800    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t" event={"ID":"54db0b92-a110-4b17-ae95-a58aa3fed9e0","Type":"ContainerStarted","Data":"2bbc039113e3c509564be20f715b0bbc1ec24f31cbb07727805107e317bd781c"}
Jul 11 23:32:15.357916 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:15.357791    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7b7c4b75dd-lgsth"]
Jul 11 23:32:15.360255 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:32:15.360229    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee947d7e_27bc_4c75_b148_a2f95295c313.slice/crio-6a34aa1e296ae5e15a594c405f3022d31921658da7c87a42a0e0c4cac92e386b WatchSource:0}: Error finding container 6a34aa1e296ae5e15a594c405f3022d31921658da7c87a42a0e0c4cac92e386b: Status 404 returned error can't find the container with id 6a34aa1e296ae5e15a594c405f3022d31921658da7c87a42a0e0c4cac92e386b
Jul 11 23:32:16.331480 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:16.330192    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth" event={"ID":"ee947d7e-27bc-4c75-b148-a2f95295c313","Type":"ContainerStarted","Data":"8b6489870d1080bb1904359ec3fdb1d8eefdefdcf2414b12989c81f7e039b0d5"}
Jul 11 23:32:16.331480 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:16.330316    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth" event={"ID":"ee947d7e-27bc-4c75-b148-a2f95295c313","Type":"ContainerStarted","Data":"6a34aa1e296ae5e15a594c405f3022d31921658da7c87a42a0e0c4cac92e386b"}
Jul 11 23:32:16.331480 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:16.330553    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:16.702282 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:16.701830    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs\") pod \"network-metrics-daemon-5qc5j\" (UID: \"7b557007-8a5a-47eb-a23e-a96ed97fb64c\") " pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:32:16.704787 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:16.704570    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 11 23:32:16.724142 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:16.720691    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b557007-8a5a-47eb-a23e-a96ed97fb64c-metrics-certs\") pod \"network-metrics-daemon-5qc5j\" (UID: \"7b557007-8a5a-47eb-a23e-a96ed97fb64c\") " pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:32:16.792772 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:16.792731    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nxtg4\""
Jul 11 23:32:16.800105 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:16.800070    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5qc5j"
Jul 11 23:32:16.803400 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:16.803364    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret\") pod \"global-pull-secret-syncer-chgrx\" (UID: \"faee1e90-99c1-4255-adb0-1d56308dbfd2\") " pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:32:16.806191 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:16.805953    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 11 23:32:16.823412 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:16.823285    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/faee1e90-99c1-4255-adb0-1d56308dbfd2-original-pull-secret\") pod \"global-pull-secret-syncer-chgrx\" (UID: \"faee1e90-99c1-4255-adb0-1d56308dbfd2\") " pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:32:16.905246 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:16.904847    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4t27\" (UniqueName: \"kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27\") pod \"network-check-target-hrfff\" (UID: \"12be71b0-f3cc-4a5c-8109-03e7f70bc99f\") " pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:32:16.907758 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:16.907521    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 11 23:32:16.918006 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:16.917810    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 11 23:32:16.941134 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:16.940198    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4t27\" (UniqueName: \"kubernetes.io/projected/12be71b0-f3cc-4a5c-8109-03e7f70bc99f-kube-api-access-r4t27\") pod \"network-check-target-hrfff\" (UID: \"12be71b0-f3cc-4a5c-8109-03e7f70bc99f\") " pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:32:17.096664 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:17.096629    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-chgrx"
Jul 11 23:32:17.106406 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:17.106375    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qb5vv\""
Jul 11 23:32:17.113990 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:17.113750    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:32:18.464250 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:18.463241    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth" podStartSLOduration=4.463216197 podStartE2EDuration="4.463216197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:32:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:16.352055246 +0000 UTC m=+65.930148023" watchObservedRunningTime="2026-07-11 23:32:18.463216197 +0000 UTC m=+68.041308973"
Jul 11 23:32:18.464250 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:18.464176    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-chgrx"]
Jul 11 23:32:18.489411 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:18.489380    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5qc5j"]
Jul 11 23:32:18.490358 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:18.490336    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hrfff"]
Jul 11 23:32:18.694897 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:32:18.694857    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaee1e90_99c1_4255_adb0_1d56308dbfd2.slice/crio-905c618855303050d15875f622cd231dac44eb473bd76698b94a858edcd9d53d WatchSource:0}: Error finding container 905c618855303050d15875f622cd231dac44eb473bd76698b94a858edcd9d53d: Status 404 returned error can't find the container with id 905c618855303050d15875f622cd231dac44eb473bd76698b94a858edcd9d53d
Jul 11 23:32:18.695828 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:32:18.695681    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12be71b0_f3cc_4a5c_8109_03e7f70bc99f.slice/crio-147754e5846180ae97dbc16645a3b95957507007e7ebb3afaa41aeebe41456e8 WatchSource:0}: Error finding container 147754e5846180ae97dbc16645a3b95957507007e7ebb3afaa41aeebe41456e8: Status 404 returned error can't find the container with id 147754e5846180ae97dbc16645a3b95957507007e7ebb3afaa41aeebe41456e8
Jul 11 23:32:18.696865 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:32:18.696792    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b557007_8a5a_47eb_a23e_a96ed97fb64c.slice/crio-d0ec55cc1faa2b16bc5654e00b5750be27e4d020ed2c3b4e3fd332ef9f40e979 WatchSource:0}: Error finding container d0ec55cc1faa2b16bc5654e00b5750be27e4d020ed2c3b4e3fd332ef9f40e979: Status 404 returned error can't find the container with id d0ec55cc1faa2b16bc5654e00b5750be27e4d020ed2c3b4e3fd332ef9f40e979
Jul 11 23:32:19.357634 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:19.357599    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl" event={"ID":"0a6cb4fa-09c9-4ebd-8cc6-041e8a65d86a","Type":"ContainerStarted","Data":"b451840f7b23e23ff3ed862da5c1d95cf26c907750fe8303ebff4adf70ba3ef1"}
Jul 11 23:32:19.359109 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:19.359081    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hrfff" event={"ID":"12be71b0-f3cc-4a5c-8109-03e7f70bc99f","Type":"ContainerStarted","Data":"147754e5846180ae97dbc16645a3b95957507007e7ebb3afaa41aeebe41456e8"}
Jul 11 23:32:19.360301 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:19.360274    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-chgrx" event={"ID":"faee1e90-99c1-4255-adb0-1d56308dbfd2","Type":"ContainerStarted","Data":"905c618855303050d15875f622cd231dac44eb473bd76698b94a858edcd9d53d"}
Jul 11 23:32:19.361571 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:19.361543    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5qc5j" event={"ID":"7b557007-8a5a-47eb-a23e-a96ed97fb64c","Type":"ContainerStarted","Data":"d0ec55cc1faa2b16bc5654e00b5750be27e4d020ed2c3b4e3fd332ef9f40e979"}
Jul 11 23:32:19.373483 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:19.373405    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8dd4b756-9p8wl" podStartSLOduration=1.840676254 podStartE2EDuration="5.373384655s" totalImagesPullingTime="3.532708401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:32:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:19.372157354 +0000 UTC m=+68.950250143" watchObservedRunningTime="2026-07-11 23:32:19.373384655 +0000 UTC m=+68.951477433"
Jul 11 23:32:22.375869 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:22.375819    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7" event={"ID":"5c5435b8-e1a0-4621-8c12-65c94bd5f8c1","Type":"ContainerStarted","Data":"90d0501c1ff979219224bcea4da5f67413c4a8281dd2c4345d8ce75a140d0190"}
Jul 11 23:32:22.376471 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:22.376099    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7"
Jul 11 23:32:22.377889 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:22.377864    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7"
Jul 11 23:32:22.379057 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:22.379030    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t" event={"ID":"54db0b92-a110-4b17-ae95-a58aa3fed9e0","Type":"ContainerStarted","Data":"a68214cf1007f1ce18cfd0a97081068a5c8c524d4129fe25eba5efbd046f1373"}
Jul 11 23:32:22.379159 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:22.379073    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t" event={"ID":"54db0b92-a110-4b17-ae95-a58aa3fed9e0","Type":"ContainerStarted","Data":"ba499ee2e711336cb5306d44e7ac420727962c90eec3b19349979fdb3c98a6be"}
Jul 11 23:32:22.379159 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:22.379093    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t" event={"ID":"54db0b92-a110-4b17-ae95-a58aa3fed9e0","Type":"ContainerStarted","Data":"5f0ccd5a118e56d6f1406c3ee64bdaa3f9651e4c2e9818cc8df562f521e9d37f"}
Jul 11 23:32:22.391694 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:22.391649    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7ff448f655-8ktw7" podStartSLOduration=2.1261656110000002 podStartE2EDuration="8.39163484s" totalImagesPullingTime="6.265469229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:32:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:22.390522564 +0000 UTC m=+71.968615343" watchObservedRunningTime="2026-07-11 23:32:22.39163484 +0000 UTC m=+71.969727616"
Jul 11 23:32:22.407379 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:22.407319    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-64768c7f89-nw27t" podStartSLOduration=2.173834362 podStartE2EDuration="8.407300533s" totalImagesPullingTime="6.233466171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:32:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:22.405887555 +0000 UTC m=+71.983980335" watchObservedRunningTime="2026-07-11 23:32:22.407300533 +0000 UTC m=+71.985393309"
Jul 11 23:32:24.390446 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:24.389692    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hrfff" event={"ID":"12be71b0-f3cc-4a5c-8109-03e7f70bc99f","Type":"ContainerStarted","Data":"1f61d2729a180b8dbbaefe449e12c15f54e924e67193b43ac6e20c1f281c3371"}
Jul 11 23:32:24.390446 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:24.389806    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:32:24.391988 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:24.391956    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5qc5j" event={"ID":"7b557007-8a5a-47eb-a23e-a96ed97fb64c","Type":"ContainerStarted","Data":"6140ffdcb82277c04e13f6a104f679055dba1e9df4df1a8c97ae2199abbae436"}
Jul 11 23:32:24.392122 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:24.392011    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5qc5j" event={"ID":"7b557007-8a5a-47eb-a23e-a96ed97fb64c","Type":"ContainerStarted","Data":"6fb43b9afdbda7cc774b3200a6a1a01b3d94e76e0ddd4a13b0972556c35b19d4"}
Jul 11 23:32:24.412047 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:24.411984    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hrfff" podStartSLOduration=68.760541508 podStartE2EDuration="1m13.411963725s" totalImagesPullingTime="4.651422217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:24.410959493 +0000 UTC m=+73.989052273" watchObservedRunningTime="2026-07-11 23:32:24.411963725 +0000 UTC m=+73.990056485"
Jul 11 23:32:24.427294 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:24.426980    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5qc5j" podStartSLOduration=68.790058742 podStartE2EDuration="1m13.42695862s" totalImagesPullingTime="4.636899878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:24.426590182 +0000 UTC m=+74.004682955" watchObservedRunningTime="2026-07-11 23:32:24.42695862 +0000 UTC m=+74.005051397"
Jul 11 23:32:24.822847 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:24.822812    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:32:25.397219 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:25.397184    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-chgrx" event={"ID":"faee1e90-99c1-4255-adb0-1d56308dbfd2","Type":"ContainerStarted","Data":"82bf193e7d519aeeb2649d2e0ff9a89492547aaf4d7d2189ffa25bf842796ca2"}
Jul 11 23:32:25.413964 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:25.413920    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-chgrx" podStartSLOduration=67.962829163 podStartE2EDuration="1m14.413903544s" totalImagesPullingTime="6.451074381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:25.413189734 +0000 UTC m=+74.991282509" watchObservedRunningTime="2026-07-11 23:32:25.413903544 +0000 UTC m=+74.991996320"
Jul 11 23:32:29.746124 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:29.746089    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zc8ks"]
Jul 11 23:32:29.749037 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:29.749017    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:29.752437 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:29.752407    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 11 23:32:29.754134 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:29.754109    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 11 23:32:29.754535 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:29.754509    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 11 23:32:29.755377 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:29.755356    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 11 23:32:29.755513 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:29.755452    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-j7gjb\""
Jul 11 23:32:29.758573 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:29.758553    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 11 23:32:29.758711 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:29.758553    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 11 23:32:29.912036 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:29.911998    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-wtmp\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:29.912036 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:29.912050    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-accelerators-collector-config\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:29.912292 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:29.912087    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-root\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:29.912292 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:29.912125    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-tls\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:29.912292 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:29.912155    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-sys\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:29.912292 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:29.912185    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-metrics-client-ca\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:29.912292 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:29.912215    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:29.912292 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:29.912255    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-textfile\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:29.912292 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:29.912277    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5cp9\" (UniqueName: \"kubernetes.io/projected/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-kube-api-access-d5cp9\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.012720 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:30.012627    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-wtmp\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.012720 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:30.012672    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-accelerators-collector-config\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.012720 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:30.012702    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-root\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.013013 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:30.012735    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-tls\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.013013 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:30.012766    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-sys\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.013013 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:30.012803    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-metrics-client-ca\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.013013 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:30.012831    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.013013 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:30.012873    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-sys\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.013013 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:30.012879    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-textfile\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.013013 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:30.012926    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5cp9\" (UniqueName: \"kubernetes.io/projected/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-kube-api-access-d5cp9\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.013377 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:30.013154    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-textfile\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.013377 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:30.013289    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-root\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.013516 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:32:30.013383    2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 11 23:32:30.013516 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:30.013421    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-accelerators-collector-config\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.013516 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:32:30.013460    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-tls podName:f27e72a2-5a95-4e89-a0a9-48e2c37b4985 nodeName:}" failed. No retries permitted until 2026-07-11 23:32:30.513422417 +0000 UTC m=+80.091534112 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-tls") pod "node-exporter-zc8ks" (UID: "f27e72a2-5a95-4e89-a0a9-48e2c37b4985") : secret "node-exporter-tls" not found
Jul 11 23:32:30.013683 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:30.013639    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-metrics-client-ca\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.013683 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:30.012833    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-wtmp\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.015642 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:30.015611    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.020973 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:30.020951    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5cp9\" (UniqueName: \"kubernetes.io/projected/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-kube-api-access-d5cp9\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.517950 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:30.517920    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-tls\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:30.518105 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:32:30.518025    2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 11 23:32:30.518105 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:32:30.518079    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-tls podName:f27e72a2-5a95-4e89-a0a9-48e2c37b4985 nodeName:}" failed. No retries permitted until 2026-07-11 23:32:31.518062579 +0000 UTC m=+81.096155334 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-tls") pod "node-exporter-zc8ks" (UID: "f27e72a2-5a95-4e89-a0a9-48e2c37b4985") : secret "node-exporter-tls" not found
Jul 11 23:32:31.526152 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:31.526114    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-tls\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:31.528885 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:31.528855    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f27e72a2-5a95-4e89-a0a9-48e2c37b4985-node-exporter-tls\") pod \"node-exporter-zc8ks\" (UID: \"f27e72a2-5a95-4e89-a0a9-48e2c37b4985\") " pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:31.559545 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:31.559506    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zc8ks"
Jul 11 23:32:31.568367 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:32:31.568339    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf27e72a2_5a95_4e89_a0a9_48e2c37b4985.slice/crio-bf8379eb3a3489bd9a85840411d1b16009b5a5e40fdbaa8d7f923a01f888ecba WatchSource:0}: Error finding container bf8379eb3a3489bd9a85840411d1b16009b5a5e40fdbaa8d7f923a01f888ecba: Status 404 returned error can't find the container with id bf8379eb3a3489bd9a85840411d1b16009b5a5e40fdbaa8d7f923a01f888ecba
Jul 11 23:32:32.424650 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:32.424612    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zc8ks" event={"ID":"f27e72a2-5a95-4e89-a0a9-48e2c37b4985","Type":"ContainerStarted","Data":"bf8379eb3a3489bd9a85840411d1b16009b5a5e40fdbaa8d7f923a01f888ecba"}
Jul 11 23:32:33.430214 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:33.430176    2569 generic.go:356] "Generic (PLEG): container finished" podID="f27e72a2-5a95-4e89-a0a9-48e2c37b4985" containerID="6a796e4c62593a57c8407e5dcda3b0de10b6f05782846c53190b6d1c36c4bb2c" exitCode=0
Jul 11 23:32:33.430612 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:33.430246    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zc8ks" event={"ID":"f27e72a2-5a95-4e89-a0a9-48e2c37b4985","Type":"ContainerDied","Data":"6a796e4c62593a57c8407e5dcda3b0de10b6f05782846c53190b6d1c36c4bb2c"}
Jul 11 23:32:34.435578 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:34.435546    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zc8ks" event={"ID":"f27e72a2-5a95-4e89-a0a9-48e2c37b4985","Type":"ContainerStarted","Data":"92d90cad3f18c043a32da94f75326b824c6dcc976d8e3f8e1711fe6c57c46933"}
Jul 11 23:32:34.435987 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:34.435595    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zc8ks" event={"ID":"f27e72a2-5a95-4e89-a0a9-48e2c37b4985","Type":"ContainerStarted","Data":"c071f4ec3ff20077e861e06632319dc29ba2e7256a26c03e219a3e9c7cfd435b"}
Jul 11 23:32:34.457325 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:34.457275    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zc8ks" podStartSLOduration=4.299474181 podStartE2EDuration="5.457260178s" totalImagesPullingTime="1.093527973s" totalInitContainerRuntime="64.258024ms" isStatefulPod=true podCreationTimestamp="2026-07-11 23:32:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:34.455561472 +0000 UTC m=+84.033654285" watchObservedRunningTime="2026-07-11 23:32:34.457260178 +0000 UTC m=+84.035353000"
Jul 11 23:32:37.344410 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:37.344377    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7b7c4b75dd-lgsth"
Jul 11 23:32:39.835181 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:39.835115    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7f664cb986-z7cww" podUID="3c1dc75e-e9b9-4f38-916c-51230cc19a38" containerName="registry" containerID="cri-o://4a89b6f2aaf6c867727b9ff78243fd34e582a4d5bda39153ea40caa0808ef8a5" gracePeriod=30
Jul 11 23:32:40.074736 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.074709    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:32:40.192940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.192863    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c1dc75e-e9b9-4f38-916c-51230cc19a38-image-registry-private-configuration\") pod \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") "
Jul 11 23:32:40.192940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.192906    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c1dc75e-e9b9-4f38-916c-51230cc19a38-ca-trust-extracted\") pod \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") "
Jul 11 23:32:40.192940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.192930    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c1dc75e-e9b9-4f38-916c-51230cc19a38-trusted-ca\") pod \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") "
Jul 11 23:32:40.193197 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.192948    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-bound-sa-token\") pod \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") "
Jul 11 23:32:40.193197 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.192978    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c1dc75e-e9b9-4f38-916c-51230cc19a38-installation-pull-secrets\") pod \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") "
Jul 11 23:32:40.193197 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.193018    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsrhl\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-kube-api-access-zsrhl\") pod \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") "
Jul 11 23:32:40.193197 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.193156    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-certificates\") pod \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") "
Jul 11 23:32:40.193375 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.193222    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls\") pod \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\" (UID: \"3c1dc75e-e9b9-4f38-916c-51230cc19a38\") "
Jul 11 23:32:40.193625 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.193599    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "3c1dc75e-e9b9-4f38-916c-51230cc19a38" (UID: "3c1dc75e-e9b9-4f38-916c-51230cc19a38"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 23:32:40.194127 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.194090    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c1dc75e-e9b9-4f38-916c-51230cc19a38-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "3c1dc75e-e9b9-4f38-916c-51230cc19a38" (UID: "3c1dc75e-e9b9-4f38-916c-51230cc19a38"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 23:32:40.195405 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.195381    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "3c1dc75e-e9b9-4f38-916c-51230cc19a38" (UID: "3c1dc75e-e9b9-4f38-916c-51230cc19a38"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 23:32:40.195573 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.195536    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c1dc75e-e9b9-4f38-916c-51230cc19a38-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "3c1dc75e-e9b9-4f38-916c-51230cc19a38" (UID: "3c1dc75e-e9b9-4f38-916c-51230cc19a38"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 23:32:40.195662 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.195624    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c1dc75e-e9b9-4f38-916c-51230cc19a38-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "3c1dc75e-e9b9-4f38-916c-51230cc19a38" (UID: "3c1dc75e-e9b9-4f38-916c-51230cc19a38"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 23:32:40.195776 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.195753    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3c1dc75e-e9b9-4f38-916c-51230cc19a38" (UID: "3c1dc75e-e9b9-4f38-916c-51230cc19a38"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 23:32:40.195844 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.195827    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-kube-api-access-zsrhl" (OuterVolumeSpecName: "kube-api-access-zsrhl") pod "3c1dc75e-e9b9-4f38-916c-51230cc19a38" (UID: "3c1dc75e-e9b9-4f38-916c-51230cc19a38"). InnerVolumeSpecName "kube-api-access-zsrhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 23:32:40.201962 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.201876    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c1dc75e-e9b9-4f38-916c-51230cc19a38-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "3c1dc75e-e9b9-4f38-916c-51230cc19a38" (UID: "3c1dc75e-e9b9-4f38-916c-51230cc19a38"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 23:32:40.294394 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.294350    2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c1dc75e-e9b9-4f38-916c-51230cc19a38-image-registry-private-configuration\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:32:40.294394 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.294385    2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c1dc75e-e9b9-4f38-916c-51230cc19a38-ca-trust-extracted\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:32:40.294394 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.294398    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c1dc75e-e9b9-4f38-916c-51230cc19a38-trusted-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:32:40.294666 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.294413    2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-bound-sa-token\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:32:40.294666 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.294445    2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c1dc75e-e9b9-4f38-916c-51230cc19a38-installation-pull-secrets\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:32:40.294666 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.294458    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsrhl\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-kube-api-access-zsrhl\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:32:40.294666 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.294471    2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-certificates\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:32:40.294666 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.294482    2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c1dc75e-e9b9-4f38-916c-51230cc19a38-registry-tls\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:32:40.460514 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.460403    2569 generic.go:356] "Generic (PLEG): container finished" podID="3c1dc75e-e9b9-4f38-916c-51230cc19a38" containerID="4a89b6f2aaf6c867727b9ff78243fd34e582a4d5bda39153ea40caa0808ef8a5" exitCode=0
Jul 11 23:32:40.460514 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.460466    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f664cb986-z7cww"
Jul 11 23:32:40.460700 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.460470    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f664cb986-z7cww" event={"ID":"3c1dc75e-e9b9-4f38-916c-51230cc19a38","Type":"ContainerDied","Data":"4a89b6f2aaf6c867727b9ff78243fd34e582a4d5bda39153ea40caa0808ef8a5"}
Jul 11 23:32:40.460700 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.460571    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f664cb986-z7cww" event={"ID":"3c1dc75e-e9b9-4f38-916c-51230cc19a38","Type":"ContainerDied","Data":"149e2091a759e912a5d4923170255f5ca104a486a648395e0731101dfc656bff"}
Jul 11 23:32:40.460700 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.460591    2569 scope.go:117] "RemoveContainer" containerID="4a89b6f2aaf6c867727b9ff78243fd34e582a4d5bda39153ea40caa0808ef8a5"
Jul 11 23:32:40.470950 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.470020    2569 scope.go:117] "RemoveContainer" containerID="4a89b6f2aaf6c867727b9ff78243fd34e582a4d5bda39153ea40caa0808ef8a5"
Jul 11 23:32:40.470950 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:32:40.470938    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a89b6f2aaf6c867727b9ff78243fd34e582a4d5bda39153ea40caa0808ef8a5\": container with ID starting with 4a89b6f2aaf6c867727b9ff78243fd34e582a4d5bda39153ea40caa0808ef8a5 not found: ID does not exist" containerID="4a89b6f2aaf6c867727b9ff78243fd34e582a4d5bda39153ea40caa0808ef8a5"
Jul 11 23:32:40.471065 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.470966    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a89b6f2aaf6c867727b9ff78243fd34e582a4d5bda39153ea40caa0808ef8a5"} err="failed to get container status \"4a89b6f2aaf6c867727b9ff78243fd34e582a4d5bda39153ea40caa0808ef8a5\": rpc error: code = NotFound desc = could not find container \"4a89b6f2aaf6c867727b9ff78243fd34e582a4d5bda39153ea40caa0808ef8a5\": container with ID starting with 4a89b6f2aaf6c867727b9ff78243fd34e582a4d5bda39153ea40caa0808ef8a5 not found: ID does not exist"
Jul 11 23:32:40.481347 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.481325    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f664cb986-z7cww"]
Jul 11 23:32:40.488057 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.488032    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7f664cb986-z7cww"]
Jul 11 23:32:40.981163 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:40.981129    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c1dc75e-e9b9-4f38-916c-51230cc19a38" path="/var/lib/kubelet/pods/3c1dc75e-e9b9-4f38-916c-51230cc19a38/volumes"
Jul 11 23:32:55.399817 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:32:55.399787    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hrfff"
Jul 11 23:33:11.313378 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:33:11.313351    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-25dj7_03eb4687-6b09-4c00-8601-4f7e11f8de4c/serve-healthcheck-canary/0.log"
Jul 11 23:35:38.565489 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.565405    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"]
Jul 11 23:35:38.565926 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.565692    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c1dc75e-e9b9-4f38-916c-51230cc19a38" containerName="registry"
Jul 11 23:35:38.565926 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.565705    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c1dc75e-e9b9-4f38-916c-51230cc19a38" containerName="registry"
Jul 11 23:35:38.565926 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.565766    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c1dc75e-e9b9-4f38-916c-51230cc19a38" containerName="registry"
Jul 11 23:35:38.567389 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.567373    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:35:38.569815 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.569785    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 11 23:35:38.569976 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.569959    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 11 23:35:38.570137 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.570037    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dzfxt\""
Jul 11 23:35:38.570137 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.570039    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 11 23:35:38.570257 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.570239    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 11 23:35:38.586442 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.585764    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"]
Jul 11 23:35:38.681414 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.681376    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e77d5c53-f8da-4726-b9da-4528dcc3db5f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d55578dd8-5z8ck\" (UID: \"e77d5c53-f8da-4726-b9da-4528dcc3db5f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:35:38.681595 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.681446    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e77d5c53-f8da-4726-b9da-4528dcc3db5f-tmp\") pod \"opendatahub-operator-controller-manager-6d55578dd8-5z8ck\" (UID: \"e77d5c53-f8da-4726-b9da-4528dcc3db5f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:35:38.681595 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.681505    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hwj8\" (UniqueName: \"kubernetes.io/projected/e77d5c53-f8da-4726-b9da-4528dcc3db5f-kube-api-access-2hwj8\") pod \"opendatahub-operator-controller-manager-6d55578dd8-5z8ck\" (UID: \"e77d5c53-f8da-4726-b9da-4528dcc3db5f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:35:38.681595 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.681588    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e77d5c53-f8da-4726-b9da-4528dcc3db5f-manifests\") pod \"opendatahub-operator-controller-manager-6d55578dd8-5z8ck\" (UID: \"e77d5c53-f8da-4726-b9da-4528dcc3db5f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:35:38.681692 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.681617    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e77d5c53-f8da-4726-b9da-4528dcc3db5f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d55578dd8-5z8ck\" (UID: \"e77d5c53-f8da-4726-b9da-4528dcc3db5f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:35:38.782849 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.782808    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e77d5c53-f8da-4726-b9da-4528dcc3db5f-manifests\") pod \"opendatahub-operator-controller-manager-6d55578dd8-5z8ck\" (UID: \"e77d5c53-f8da-4726-b9da-4528dcc3db5f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:35:38.782849 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.782852    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e77d5c53-f8da-4726-b9da-4528dcc3db5f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d55578dd8-5z8ck\" (UID: \"e77d5c53-f8da-4726-b9da-4528dcc3db5f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:35:38.783049 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.782882    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e77d5c53-f8da-4726-b9da-4528dcc3db5f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d55578dd8-5z8ck\" (UID: \"e77d5c53-f8da-4726-b9da-4528dcc3db5f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:35:38.783049 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.782925    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e77d5c53-f8da-4726-b9da-4528dcc3db5f-tmp\") pod \"opendatahub-operator-controller-manager-6d55578dd8-5z8ck\" (UID: \"e77d5c53-f8da-4726-b9da-4528dcc3db5f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:35:38.783049 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.782951    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hwj8\" (UniqueName: \"kubernetes.io/projected/e77d5c53-f8da-4726-b9da-4528dcc3db5f-kube-api-access-2hwj8\") pod \"opendatahub-operator-controller-manager-6d55578dd8-5z8ck\" (UID: \"e77d5c53-f8da-4726-b9da-4528dcc3db5f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:35:38.783307 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.783287    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e77d5c53-f8da-4726-b9da-4528dcc3db5f-manifests\") pod \"opendatahub-operator-controller-manager-6d55578dd8-5z8ck\" (UID: \"e77d5c53-f8da-4726-b9da-4528dcc3db5f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:35:38.785326 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.785299    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e77d5c53-f8da-4726-b9da-4528dcc3db5f-tmp\") pod \"opendatahub-operator-controller-manager-6d55578dd8-5z8ck\" (UID: \"e77d5c53-f8da-4726-b9da-4528dcc3db5f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:35:38.785493 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.785484    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e77d5c53-f8da-4726-b9da-4528dcc3db5f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d55578dd8-5z8ck\" (UID: \"e77d5c53-f8da-4726-b9da-4528dcc3db5f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:35:38.785614 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.785592    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e77d5c53-f8da-4726-b9da-4528dcc3db5f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d55578dd8-5z8ck\" (UID: \"e77d5c53-f8da-4726-b9da-4528dcc3db5f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:35:38.790933 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.790894    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hwj8\" (UniqueName: \"kubernetes.io/projected/e77d5c53-f8da-4726-b9da-4528dcc3db5f-kube-api-access-2hwj8\") pod \"opendatahub-operator-controller-manager-6d55578dd8-5z8ck\" (UID: \"e77d5c53-f8da-4726-b9da-4528dcc3db5f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:35:38.877701 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:38.877617    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:35:39.009969 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:39.009938    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"]
Jul 11 23:35:39.013692 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:35:39.013668    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode77d5c53_f8da_4726_b9da_4528dcc3db5f.slice/crio-a755e8e42989bf2ae8a74d6bf36a2b2ac2ea65a4c0863f98798835d44e9c9340 WatchSource:0}: Error finding container a755e8e42989bf2ae8a74d6bf36a2b2ac2ea65a4c0863f98798835d44e9c9340: Status 404 returned error can't find the container with id a755e8e42989bf2ae8a74d6bf36a2b2ac2ea65a4c0863f98798835d44e9c9340
Jul 11 23:35:39.052461 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:39.052396    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck" event={"ID":"e77d5c53-f8da-4726-b9da-4528dcc3db5f","Type":"ContainerStarted","Data":"a755e8e42989bf2ae8a74d6bf36a2b2ac2ea65a4c0863f98798835d44e9c9340"}
Jul 11 23:35:42.065637 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:42.065600    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck" event={"ID":"e77d5c53-f8da-4726-b9da-4528dcc3db5f","Type":"ContainerStarted","Data":"3500f791b7ffa28e034dbb497b02d71312fe2e44633aa6507d17bf9673c04009"}
Jul 11 23:35:44.073764 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:44.073730    2569 generic.go:356] "Generic (PLEG): container finished" podID="e77d5c53-f8da-4726-b9da-4528dcc3db5f" containerID="3500f791b7ffa28e034dbb497b02d71312fe2e44633aa6507d17bf9673c04009" exitCode=0
Jul 11 23:35:44.074199 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:44.073812    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck" event={"ID":"e77d5c53-f8da-4726-b9da-4528dcc3db5f","Type":"ContainerDied","Data":"3500f791b7ffa28e034dbb497b02d71312fe2e44633aa6507d17bf9673c04009"}
Jul 11 23:35:45.078396 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:45.078360    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck" event={"ID":"e77d5c53-f8da-4726-b9da-4528dcc3db5f","Type":"ContainerStarted","Data":"3f25805bc6976a3fa82fde52247a584302ddf96e00f3beb44cdc131714bf35fd"}
Jul 11 23:35:45.078842 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:45.078592    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:35:45.101583 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:45.101528    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck" podStartSLOduration=2.823958074 podStartE2EDuration="7.10151391s" totalImagesPullingTime="3.197155572s" totalInitContainerRuntime="1.080400264s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:35:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:35:45.099733465 +0000 UTC m=+274.677826241" watchObservedRunningTime="2026-07-11 23:35:45.10151391 +0000 UTC m=+274.679606752"
Jul 11 23:35:56.084386 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:35:56.084354    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-5z8ck"
Jul 11 23:36:03.506152 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.506115    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7bd58cf797-5gb9m"]
Jul 11 23:36:03.509308 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.509283    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bd58cf797-5gb9m"
Jul 11 23:36:03.511793 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.511770    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 11 23:36:03.511793 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.511789    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 11 23:36:03.512932 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.512905    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 11 23:36:03.513047 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.512933    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-j98pt\""
Jul 11 23:36:03.513047 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.512908    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 11 23:36:03.519617 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.519592    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bd58cf797-5gb9m"]
Jul 11 23:36:03.559898 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.559857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e1f7f37-08f8-4664-8d07-10b594f04655-tls-certs\") pod \"kube-auth-proxy-7bd58cf797-5gb9m\" (UID: \"8e1f7f37-08f8-4664-8d07-10b594f04655\") " pod="openshift-ingress/kube-auth-proxy-7bd58cf797-5gb9m"
Jul 11 23:36:03.559898 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.559908    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e1f7f37-08f8-4664-8d07-10b594f04655-tmp\") pod \"kube-auth-proxy-7bd58cf797-5gb9m\" (UID: \"8e1f7f37-08f8-4664-8d07-10b594f04655\") " pod="openshift-ingress/kube-auth-proxy-7bd58cf797-5gb9m"
Jul 11 23:36:03.560112 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.559928    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s67vj\" (UniqueName: \"kubernetes.io/projected/8e1f7f37-08f8-4664-8d07-10b594f04655-kube-api-access-s67vj\") pod \"kube-auth-proxy-7bd58cf797-5gb9m\" (UID: \"8e1f7f37-08f8-4664-8d07-10b594f04655\") " pod="openshift-ingress/kube-auth-proxy-7bd58cf797-5gb9m"
Jul 11 23:36:03.661134 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.661092    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e1f7f37-08f8-4664-8d07-10b594f04655-tls-certs\") pod \"kube-auth-proxy-7bd58cf797-5gb9m\" (UID: \"8e1f7f37-08f8-4664-8d07-10b594f04655\") " pod="openshift-ingress/kube-auth-proxy-7bd58cf797-5gb9m"
Jul 11 23:36:03.661312 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.661143    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e1f7f37-08f8-4664-8d07-10b594f04655-tmp\") pod \"kube-auth-proxy-7bd58cf797-5gb9m\" (UID: \"8e1f7f37-08f8-4664-8d07-10b594f04655\") " pod="openshift-ingress/kube-auth-proxy-7bd58cf797-5gb9m"
Jul 11 23:36:03.661312 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.661165    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s67vj\" (UniqueName: \"kubernetes.io/projected/8e1f7f37-08f8-4664-8d07-10b594f04655-kube-api-access-s67vj\") pod \"kube-auth-proxy-7bd58cf797-5gb9m\" (UID: \"8e1f7f37-08f8-4664-8d07-10b594f04655\") " pod="openshift-ingress/kube-auth-proxy-7bd58cf797-5gb9m"
Jul 11 23:36:03.663409 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.663373    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e1f7f37-08f8-4664-8d07-10b594f04655-tmp\") pod \"kube-auth-proxy-7bd58cf797-5gb9m\" (UID: \"8e1f7f37-08f8-4664-8d07-10b594f04655\") " pod="openshift-ingress/kube-auth-proxy-7bd58cf797-5gb9m"
Jul 11 23:36:03.663609 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.663588    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e1f7f37-08f8-4664-8d07-10b594f04655-tls-certs\") pod \"kube-auth-proxy-7bd58cf797-5gb9m\" (UID: \"8e1f7f37-08f8-4664-8d07-10b594f04655\") " pod="openshift-ingress/kube-auth-proxy-7bd58cf797-5gb9m"
Jul 11 23:36:03.671685 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.671656    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s67vj\" (UniqueName: \"kubernetes.io/projected/8e1f7f37-08f8-4664-8d07-10b594f04655-kube-api-access-s67vj\") pod \"kube-auth-proxy-7bd58cf797-5gb9m\" (UID: \"8e1f7f37-08f8-4664-8d07-10b594f04655\") " pod="openshift-ingress/kube-auth-proxy-7bd58cf797-5gb9m"
Jul 11 23:36:03.819175 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.819143    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bd58cf797-5gb9m"
Jul 11 23:36:03.958295 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:03.958271    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bd58cf797-5gb9m"]
Jul 11 23:36:03.960713 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:36:03.960689    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e1f7f37_08f8_4664_8d07_10b594f04655.slice/crio-4268dff3e869a43ba3752d3fe3fd1a08514f6af87915c6b059f2e3446a9826ea WatchSource:0}: Error finding container 4268dff3e869a43ba3752d3fe3fd1a08514f6af87915c6b059f2e3446a9826ea: Status 404 returned error can't find the container with id 4268dff3e869a43ba3752d3fe3fd1a08514f6af87915c6b059f2e3446a9826ea
Jul 11 23:36:04.143479 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:04.143365    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bd58cf797-5gb9m" event={"ID":"8e1f7f37-08f8-4664-8d07-10b594f04655","Type":"ContainerStarted","Data":"4268dff3e869a43ba3752d3fe3fd1a08514f6af87915c6b059f2e3446a9826ea"}
Jul 11 23:36:06.654222 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.654190    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"]
Jul 11 23:36:06.657383 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.657366    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.662141 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.662117    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 11 23:36:06.662263 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.662140    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 11 23:36:06.662263 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.662177    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bvm5c\""
Jul 11 23:36:06.662485 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.662451    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 11 23:36:06.662603 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.662538    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 11 23:36:06.680146 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.680118    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"]
Jul 11 23:36:06.687911 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.687887    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d244dd8-877c-4b19-9a57-254eaff8b596-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.688031 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.687921    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9d244dd8-877c-4b19-9a57-254eaff8b596-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.688031 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.687943    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnpxw\" (UniqueName: \"kubernetes.io/projected/9d244dd8-877c-4b19-9a57-254eaff8b596-kube-api-access-hnpxw\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.688031 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.687968    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9d244dd8-877c-4b19-9a57-254eaff8b596-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.688031 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.688016    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d244dd8-877c-4b19-9a57-254eaff8b596-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.688213 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.688045    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9d244dd8-877c-4b19-9a57-254eaff8b596-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.688213 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.688109    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9d244dd8-877c-4b19-9a57-254eaff8b596-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.788886 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.788789    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d244dd8-877c-4b19-9a57-254eaff8b596-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.788886 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.788828    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9d244dd8-877c-4b19-9a57-254eaff8b596-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.788886 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.788845    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnpxw\" (UniqueName: \"kubernetes.io/projected/9d244dd8-877c-4b19-9a57-254eaff8b596-kube-api-access-hnpxw\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.788886 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.788867    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9d244dd8-877c-4b19-9a57-254eaff8b596-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.789195 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.788902    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d244dd8-877c-4b19-9a57-254eaff8b596-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.789195 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.788940    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9d244dd8-877c-4b19-9a57-254eaff8b596-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.789195 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.788978    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9d244dd8-877c-4b19-9a57-254eaff8b596-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.789637 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.789608    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9d244dd8-877c-4b19-9a57-254eaff8b596-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.791359 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.791334    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9d244dd8-877c-4b19-9a57-254eaff8b596-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.791505 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.791372    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9d244dd8-877c-4b19-9a57-254eaff8b596-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.791573 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.791555    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d244dd8-877c-4b19-9a57-254eaff8b596-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.791632 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.791587    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9d244dd8-877c-4b19-9a57-254eaff8b596-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.796899 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.796863    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d244dd8-877c-4b19-9a57-254eaff8b596-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.797308 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.797291    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnpxw\" (UniqueName: \"kubernetes.io/projected/9d244dd8-877c-4b19-9a57-254eaff8b596-kube-api-access-hnpxw\") pod \"istiod-openshift-gateway-94bb8fbfd-vdl2s\" (UID: \"9d244dd8-877c-4b19-9a57-254eaff8b596\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:06.965901 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:06.965867    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:07.115155 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:07.115131    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"]
Jul 11 23:36:07.117237 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:36:07.117211    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d244dd8_877c_4b19_9a57_254eaff8b596.slice/crio-2f48b884b235b5a417685b3fc69f3197111b9213079f117e05895cdc84ea11d3 WatchSource:0}: Error finding container 2f48b884b235b5a417685b3fc69f3197111b9213079f117e05895cdc84ea11d3: Status 404 returned error can't find the container with id 2f48b884b235b5a417685b3fc69f3197111b9213079f117e05895cdc84ea11d3
Jul 11 23:36:07.156189 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:07.156156    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bd58cf797-5gb9m" event={"ID":"8e1f7f37-08f8-4664-8d07-10b594f04655","Type":"ContainerStarted","Data":"dc48cb8c03f9db8c15fe61e59eab5b337348af77a58912687893e66c05526bec"}
Jul 11 23:36:07.157404 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:07.157377    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s" event={"ID":"9d244dd8-877c-4b19-9a57-254eaff8b596","Type":"ContainerStarted","Data":"2f48b884b235b5a417685b3fc69f3197111b9213079f117e05895cdc84ea11d3"}
Jul 11 23:36:07.185067 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:07.185024    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7bd58cf797-5gb9m" podStartSLOduration=1.653090792 podStartE2EDuration="4.185009008s" totalImagesPullingTime="2.531918216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:36:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:36:07.183613998 +0000 UTC m=+296.761706776" watchObservedRunningTime="2026-07-11 23:36:07.185009008 +0000 UTC m=+296.763101783"
Jul 11 23:36:08.188686 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.188645    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qmhwn"]
Jul 11 23:36:08.195346 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.195300    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmhwn"
Jul 11 23:36:08.203954 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.203515    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-svxl8\""
Jul 11 23:36:08.204467 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.204256    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 11 23:36:08.217993 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.217956    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qmhwn"]
Jul 11 23:36:08.217993 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.218268    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pt2vk"]
Jul 11 23:36:08.223707 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.223563    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pt2vk"
Jul 11 23:36:08.224162 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.224145    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pt2vk"]
Jul 11 23:36:08.230583 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.230275    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-tc656\""
Jul 11 23:36:08.231198 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.230554    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 11 23:36:08.302739 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.302702    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bad0275-0b31-4253-a626-88f6c74b1e91-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pt2vk\" (UID: \"9bad0275-0b31-4253-a626-88f6c74b1e91\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pt2vk"
Jul 11 23:36:08.302915 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.302772    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt8pt\" (UniqueName: \"kubernetes.io/projected/59861bfe-ef22-4152-b1c1-ea4f3eba7cf4-kube-api-access-bt8pt\") pod \"kserve-controller-manager-d6f4b65b8-qmhwn\" (UID: \"59861bfe-ef22-4152-b1c1-ea4f3eba7cf4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmhwn"
Jul 11 23:36:08.302991 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.302953    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59861bfe-ef22-4152-b1c1-ea4f3eba7cf4-cert\") pod \"kserve-controller-manager-d6f4b65b8-qmhwn\" (UID: \"59861bfe-ef22-4152-b1c1-ea4f3eba7cf4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmhwn"
Jul 11 23:36:08.303044 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.302987    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9x6h\" (UniqueName: \"kubernetes.io/projected/9bad0275-0b31-4253-a626-88f6c74b1e91-kube-api-access-l9x6h\") pod \"llmisvc-controller-manager-777c6bd7d5-pt2vk\" (UID: \"9bad0275-0b31-4253-a626-88f6c74b1e91\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pt2vk"
Jul 11 23:36:08.403765 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.403734    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bad0275-0b31-4253-a626-88f6c74b1e91-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pt2vk\" (UID: \"9bad0275-0b31-4253-a626-88f6c74b1e91\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pt2vk"
Jul 11 23:36:08.403946 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.403781    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt8pt\" (UniqueName: \"kubernetes.io/projected/59861bfe-ef22-4152-b1c1-ea4f3eba7cf4-kube-api-access-bt8pt\") pod \"kserve-controller-manager-d6f4b65b8-qmhwn\" (UID: \"59861bfe-ef22-4152-b1c1-ea4f3eba7cf4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmhwn"
Jul 11 23:36:08.403946 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.403822    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59861bfe-ef22-4152-b1c1-ea4f3eba7cf4-cert\") pod \"kserve-controller-manager-d6f4b65b8-qmhwn\" (UID: \"59861bfe-ef22-4152-b1c1-ea4f3eba7cf4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmhwn"
Jul 11 23:36:08.403946 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.403839    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9x6h\" (UniqueName: \"kubernetes.io/projected/9bad0275-0b31-4253-a626-88f6c74b1e91-kube-api-access-l9x6h\") pod \"llmisvc-controller-manager-777c6bd7d5-pt2vk\" (UID: \"9bad0275-0b31-4253-a626-88f6c74b1e91\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pt2vk"
Jul 11 23:36:08.403946 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:36:08.403892    2569 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 11 23:36:08.404161 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:36:08.403957    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9bad0275-0b31-4253-a626-88f6c74b1e91-cert podName:9bad0275-0b31-4253-a626-88f6c74b1e91 nodeName:}" failed. No retries permitted until 2026-07-11 23:36:08.90394049 +0000 UTC m=+298.482033243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9bad0275-0b31-4253-a626-88f6c74b1e91-cert") pod "llmisvc-controller-manager-777c6bd7d5-pt2vk" (UID: "9bad0275-0b31-4253-a626-88f6c74b1e91") : secret "llmisvc-webhook-server-cert" not found
Jul 11 23:36:08.406539 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.406514    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59861bfe-ef22-4152-b1c1-ea4f3eba7cf4-cert\") pod \"kserve-controller-manager-d6f4b65b8-qmhwn\" (UID: \"59861bfe-ef22-4152-b1c1-ea4f3eba7cf4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmhwn"
Jul 11 23:36:08.427041 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.427010    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt8pt\" (UniqueName: \"kubernetes.io/projected/59861bfe-ef22-4152-b1c1-ea4f3eba7cf4-kube-api-access-bt8pt\") pod \"kserve-controller-manager-d6f4b65b8-qmhwn\" (UID: \"59861bfe-ef22-4152-b1c1-ea4f3eba7cf4\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmhwn"
Jul 11 23:36:08.427041 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.427030    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9x6h\" (UniqueName: \"kubernetes.io/projected/9bad0275-0b31-4253-a626-88f6c74b1e91-kube-api-access-l9x6h\") pod \"llmisvc-controller-manager-777c6bd7d5-pt2vk\" (UID: \"9bad0275-0b31-4253-a626-88f6c74b1e91\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pt2vk"
Jul 11 23:36:08.513586 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.513510    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmhwn"
Jul 11 23:36:08.658775 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.658744    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qmhwn"]
Jul 11 23:36:08.662610 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:36:08.662577    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59861bfe_ef22_4152_b1c1_ea4f3eba7cf4.slice/crio-f56b052fd645d2ae8833947e39fef5a8fdd94ab6661bc72aa54e8e9ec7f0d2dd WatchSource:0}: Error finding container f56b052fd645d2ae8833947e39fef5a8fdd94ab6661bc72aa54e8e9ec7f0d2dd: Status 404 returned error can't find the container with id f56b052fd645d2ae8833947e39fef5a8fdd94ab6661bc72aa54e8e9ec7f0d2dd
Jul 11 23:36:08.908220 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.908186    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bad0275-0b31-4253-a626-88f6c74b1e91-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pt2vk\" (UID: \"9bad0275-0b31-4253-a626-88f6c74b1e91\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pt2vk"
Jul 11 23:36:08.910837 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:08.910810    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bad0275-0b31-4253-a626-88f6c74b1e91-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pt2vk\" (UID: \"9bad0275-0b31-4253-a626-88f6c74b1e91\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pt2vk"
Jul 11 23:36:09.138854 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:09.138814    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pt2vk"
Jul 11 23:36:09.167856 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:09.167780    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmhwn" event={"ID":"59861bfe-ef22-4152-b1c1-ea4f3eba7cf4","Type":"ContainerStarted","Data":"f56b052fd645d2ae8833947e39fef5a8fdd94ab6661bc72aa54e8e9ec7f0d2dd"}
Jul 11 23:36:09.615365 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:09.615314    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:36:09.615723 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:09.615411    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:36:09.707570 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:09.707542    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pt2vk"]
Jul 11 23:36:09.709722 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:36:09.709697    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9bad0275_0b31_4253_a626_88f6c74b1e91.slice/crio-d901fea47e9607723807bc0dd553618a7c3d74f2d3eb353ffa3fa18ba11ce53b WatchSource:0}: Error finding container d901fea47e9607723807bc0dd553618a7c3d74f2d3eb353ffa3fa18ba11ce53b: Status 404 returned error can't find the container with id d901fea47e9607723807bc0dd553618a7c3d74f2d3eb353ffa3fa18ba11ce53b
Jul 11 23:36:10.174185 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:10.174134    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s" event={"ID":"9d244dd8-877c-4b19-9a57-254eaff8b596","Type":"ContainerStarted","Data":"8f77157ad31ed367476ddf1d151aab0d6042485f002264b3b77cb91c753a8cc0"}
Jul 11 23:36:10.174476 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:10.174454    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:10.177054 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:10.176972    2569 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-vdl2s container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 11 23:36:10.177054 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:10.177022    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s" podUID="9d244dd8-877c-4b19-9a57-254eaff8b596" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 11 23:36:10.177871 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:10.177129    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pt2vk" event={"ID":"9bad0275-0b31-4253-a626-88f6c74b1e91","Type":"ContainerStarted","Data":"d901fea47e9607723807bc0dd553618a7c3d74f2d3eb353ffa3fa18ba11ce53b"}
Jul 11 23:36:10.205679 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:10.203999    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s" podStartSLOduration=1.708434917 podStartE2EDuration="4.203979284s" totalImagesPullingTime="2.495544367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:36:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:36:10.202051839 +0000 UTC m=+299.780144616" watchObservedRunningTime="2026-07-11 23:36:10.203979284 +0000 UTC m=+299.782072062"
Jul 11 23:36:11.011997 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.011648    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"]
Jul 11 23:36:11.018184 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.016590    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.021060 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.020958    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-dt4j9\""
Jul 11 23:36:11.046702 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.046671    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"]
Jul 11 23:36:11.129495 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.129386    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8kvh\" (UniqueName: \"kubernetes.io/projected/8b346891-a01d-4b8e-8246-afa554b6568b-kube-api-access-d8kvh\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.129495 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.129461    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b346891-a01d-4b8e-8246-afa554b6568b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.129751 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.129525    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b346891-a01d-4b8e-8246-afa554b6568b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.129751 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.129563    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b346891-a01d-4b8e-8246-afa554b6568b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.129751 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.129651    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b346891-a01d-4b8e-8246-afa554b6568b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.129751 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.129704    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b346891-a01d-4b8e-8246-afa554b6568b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.129751 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.129748    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b346891-a01d-4b8e-8246-afa554b6568b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.130000 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.129838    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b346891-a01d-4b8e-8246-afa554b6568b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.130000 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.129874    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b346891-a01d-4b8e-8246-afa554b6568b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.184641 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.184277    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-vdl2s"
Jul 11 23:36:11.230989 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.230950    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b346891-a01d-4b8e-8246-afa554b6568b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.231161 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.231035    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b346891-a01d-4b8e-8246-afa554b6568b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.231161 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.231078    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b346891-a01d-4b8e-8246-afa554b6568b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.231161 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.231126    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8kvh\" (UniqueName: \"kubernetes.io/projected/8b346891-a01d-4b8e-8246-afa554b6568b-kube-api-access-d8kvh\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.231329 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.231161    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b346891-a01d-4b8e-8246-afa554b6568b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.231329 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.231198    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b346891-a01d-4b8e-8246-afa554b6568b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.231329 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.231225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b346891-a01d-4b8e-8246-afa554b6568b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.231500 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.231336    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b346891-a01d-4b8e-8246-afa554b6568b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.231500 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.231387    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b346891-a01d-4b8e-8246-afa554b6568b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.233404 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.232323    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b346891-a01d-4b8e-8246-afa554b6568b-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.233404 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.232661    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b346891-a01d-4b8e-8246-afa554b6568b-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.233404 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.232663    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b346891-a01d-4b8e-8246-afa554b6568b-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.233404 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.232881    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b346891-a01d-4b8e-8246-afa554b6568b-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.233404 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.233125    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b346891-a01d-4b8e-8246-afa554b6568b-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.235684 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.235611    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b346891-a01d-4b8e-8246-afa554b6568b-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.238269 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.238222    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b346891-a01d-4b8e-8246-afa554b6568b-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.252169 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.252106    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8kvh\" (UniqueName: \"kubernetes.io/projected/8b346891-a01d-4b8e-8246-afa554b6568b-kube-api-access-d8kvh\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.252169 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.252139    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b346891-a01d-4b8e-8246-afa554b6568b-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4s85pq\" (UID: \"8b346891-a01d-4b8e-8246-afa554b6568b\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.329547 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.329512    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:11.758690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.758647    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 11 23:36:11.760302 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.760279    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 11 23:36:11.762058 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.762037    2569 kubelet.go:1648] "Image garbage collection succeeded"
Jul 11 23:36:11.866618 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:11.866582    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"]
Jul 11 23:36:11.870348 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:36:11.870324    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b346891_a01d_4b8e_8246_afa554b6568b.slice/crio-69cf87cbb64b596415652d3bcc590ed6ccbfda3e3c3017cb50d2c95a94f4cc0a WatchSource:0}: Error finding container 69cf87cbb64b596415652d3bcc590ed6ccbfda3e3c3017cb50d2c95a94f4cc0a: Status 404 returned error can't find the container with id 69cf87cbb64b596415652d3bcc590ed6ccbfda3e3c3017cb50d2c95a94f4cc0a
Jul 11 23:36:12.189561 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:12.189523    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmhwn" event={"ID":"59861bfe-ef22-4152-b1c1-ea4f3eba7cf4","Type":"ContainerStarted","Data":"48186fb16d3a2ca979fbd1015d21a107f02c57ca3cac4846c36c94611c467379"}
Jul 11 23:36:12.190030 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:12.189652    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmhwn"
Jul 11 23:36:12.191233 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:12.191204    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq" event={"ID":"8b346891-a01d-4b8e-8246-afa554b6568b","Type":"ContainerStarted","Data":"69cf87cbb64b596415652d3bcc590ed6ccbfda3e3c3017cb50d2c95a94f4cc0a"}
Jul 11 23:36:12.208643 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:12.208585    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmhwn" podStartSLOduration=1.143555894 podStartE2EDuration="4.208566085s" totalImagesPullingTime="3.065010191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:36:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:36:12.207480592 +0000 UTC m=+301.785573370" watchObservedRunningTime="2026-07-11 23:36:12.208566085 +0000 UTC m=+301.786658862"
Jul 11 23:36:13.198273 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:13.198240    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pt2vk" event={"ID":"9bad0275-0b31-4253-a626-88f6c74b1e91","Type":"ContainerStarted","Data":"5827abf3376e635c20cf9e9198c2c23635eca490d62717be74346b534f0e9582"}
Jul 11 23:36:13.198667 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:13.198562    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pt2vk"
Jul 11 23:36:16.731067 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:16.731034    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:36:16.731351 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:16.731098    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:36:16.731351 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:16.731126    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:36:17.217502 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:17.217468    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq" event={"ID":"8b346891-a01d-4b8e-8246-afa554b6568b","Type":"ContainerStarted","Data":"6855fc96898bd030af3362bad7e6e9768de31dfa287880135558d3a8253cd5b8"}
Jul 11 23:36:17.237760 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:17.237712    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq" podStartSLOduration=2.37957618 podStartE2EDuration="7.237696972s" totalImagesPullingTime="4.858120792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:36:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:36:17.23590227 +0000 UTC m=+306.813995046" watchObservedRunningTime="2026-07-11 23:36:17.237696972 +0000 UTC m=+306.815789747"
Jul 11 23:36:17.237940 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:17.237922    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pt2vk" podStartSLOduration=6.459832347 podStartE2EDuration="9.237918387s" totalImagesPullingTime="2.77808604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:36:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:36:13.213233657 +0000 UTC m=+302.791326434" watchObservedRunningTime="2026-07-11 23:36:17.237918387 +0000 UTC m=+306.816011198"
Jul 11 23:36:17.330093 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:17.330056    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:17.334541 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:17.334517    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:18.221489 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:18.221458    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:18.222537 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:18.222512    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4s85pq"
Jul 11 23:36:43.202854 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:43.202822    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qmhwn"
Jul 11 23:36:45.210188 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:36:45.210157    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pt2vk"
Jul 11 23:37:32.384482 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:37:32.384403    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ffzdw"]
Jul 11 23:37:32.387792 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:37:32.387776    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ffzdw"
Jul 11 23:37:32.390604 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:37:32.390581    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 11 23:37:32.390716 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:37:32.390695    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-jbnh5\""
Jul 11 23:37:32.390756 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:37:32.390697    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 11 23:37:32.391725 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:37:32.391709    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 11 23:37:32.399609 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:37:32.399590    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ffzdw"]
Jul 11 23:37:32.493244 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:37:32.493196    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljb7n\" (UniqueName: \"kubernetes.io/projected/69e4114c-5e34-4a82-a58b-20456e9c9ace-kube-api-access-ljb7n\") pod \"dns-operator-controller-manager-864dd78c4c-ffzdw\" (UID: \"69e4114c-5e34-4a82-a58b-20456e9c9ace\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ffzdw"
Jul 11 23:37:32.594326 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:37:32.594287    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljb7n\" (UniqueName: \"kubernetes.io/projected/69e4114c-5e34-4a82-a58b-20456e9c9ace-kube-api-access-ljb7n\") pod \"dns-operator-controller-manager-864dd78c4c-ffzdw\" (UID: \"69e4114c-5e34-4a82-a58b-20456e9c9ace\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ffzdw"
Jul 11 23:37:32.603842 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:37:32.603814    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljb7n\" (UniqueName: \"kubernetes.io/projected/69e4114c-5e34-4a82-a58b-20456e9c9ace-kube-api-access-ljb7n\") pod \"dns-operator-controller-manager-864dd78c4c-ffzdw\" (UID: \"69e4114c-5e34-4a82-a58b-20456e9c9ace\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ffzdw"
Jul 11 23:37:32.697310 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:37:32.697216    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ffzdw"
Jul 11 23:37:32.824756 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:37:32.824732    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ffzdw"]
Jul 11 23:37:32.827302 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:37:32.827274    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69e4114c_5e34_4a82_a58b_20456e9c9ace.slice/crio-38fd9593cef5024e0562c672dde90b1f5d9eb854428ae16768c9c37e27291f06 WatchSource:0}: Error finding container 38fd9593cef5024e0562c672dde90b1f5d9eb854428ae16768c9c37e27291f06: Status 404 returned error can't find the container with id 38fd9593cef5024e0562c672dde90b1f5d9eb854428ae16768c9c37e27291f06
Jul 11 23:37:32.829338 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:37:32.829322    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 11 23:37:33.513865 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:37:33.513825    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ffzdw" event={"ID":"69e4114c-5e34-4a82-a58b-20456e9c9ace","Type":"ContainerStarted","Data":"38fd9593cef5024e0562c672dde90b1f5d9eb854428ae16768c9c37e27291f06"}
Jul 11 23:37:35.525234 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:37:35.525202    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ffzdw" event={"ID":"69e4114c-5e34-4a82-a58b-20456e9c9ace","Type":"ContainerStarted","Data":"93c47678e61480060bee0ac815aa7417ab872acfe616a32b6f592725fbe193e0"}
Jul 11 23:37:35.525635 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:37:35.525350    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ffzdw"
Jul 11 23:37:35.545932 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:37:35.545877    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ffzdw" podStartSLOduration=1.7765037719999999 podStartE2EDuration="3.545862281s" totalImagesPullingTime="1.769358509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:37:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:37:35.544256596 +0000 UTC m=+385.122349373" watchObservedRunningTime="2026-07-11 23:37:35.545862281 +0000 UTC m=+385.123955056"
Jul 11 23:37:46.531751 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:37:46.531720    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ffzdw"
Jul 11 23:38:17.025382 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.025347    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"]
Jul 11 23:38:17.031775 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.031754    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.034319 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.034297    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-58wrb\""
Jul 11 23:38:17.040657 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.040633    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"]
Jul 11 23:38:17.061877 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.061845    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73e09177-7cd0-493f-841d-252842535a58-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.061877 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.061880    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73e09177-7cd0-493f-841d-252842535a58-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.062067 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.061912    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73e09177-7cd0-493f-841d-252842535a58-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.062067 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.061938    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73e09177-7cd0-493f-841d-252842535a58-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.062067 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.061995    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73e09177-7cd0-493f-841d-252842535a58-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.062067 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.062049    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73e09177-7cd0-493f-841d-252842535a58-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.062191 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.062067    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8ggd\" (UniqueName: \"kubernetes.io/projected/73e09177-7cd0-493f-841d-252842535a58-kube-api-access-n8ggd\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.062191 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.062086    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73e09177-7cd0-493f-841d-252842535a58-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.062191 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.062103    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73e09177-7cd0-493f-841d-252842535a58-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.162992 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.162962    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73e09177-7cd0-493f-841d-252842535a58-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.163168 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.162999    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8ggd\" (UniqueName: \"kubernetes.io/projected/73e09177-7cd0-493f-841d-252842535a58-kube-api-access-n8ggd\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.163168 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.163022    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73e09177-7cd0-493f-841d-252842535a58-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.163168 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.163042    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73e09177-7cd0-493f-841d-252842535a58-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.163168 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.163092    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73e09177-7cd0-493f-841d-252842535a58-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.163168 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.163113    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73e09177-7cd0-493f-841d-252842535a58-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.163168 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.163161    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73e09177-7cd0-493f-841d-252842535a58-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.163422 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.163184    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73e09177-7cd0-493f-841d-252842535a58-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.163422 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.163209    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73e09177-7cd0-493f-841d-252842535a58-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.163539 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.163512    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73e09177-7cd0-493f-841d-252842535a58-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.163613 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.163589    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73e09177-7cd0-493f-841d-252842535a58-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.163768 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.163744    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73e09177-7cd0-493f-841d-252842535a58-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.163886 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.163752    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73e09177-7cd0-493f-841d-252842535a58-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.163946 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.163824    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73e09177-7cd0-493f-841d-252842535a58-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.165493 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.165467    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73e09177-7cd0-493f-841d-252842535a58-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.165701 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.165682    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73e09177-7cd0-493f-841d-252842535a58-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.170333 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.170306    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8ggd\" (UniqueName: \"kubernetes.io/projected/73e09177-7cd0-493f-841d-252842535a58-kube-api-access-n8ggd\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.170944 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.170926    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73e09177-7cd0-493f-841d-252842535a58-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-hz4g6\" (UID: \"73e09177-7cd0-493f-841d-252842535a58\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.342282 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.342251    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:17.476408 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.476370    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"]
Jul 11 23:38:17.479543 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:38:17.479514    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73e09177_7cd0_493f_841d_252842535a58.slice/crio-7b4a68f0a4a0dbf7bec7c8c311ea7a2f0dacfe786a280d83a949d6ac5682aef7 WatchSource:0}: Error finding container 7b4a68f0a4a0dbf7bec7c8c311ea7a2f0dacfe786a280d83a949d6ac5682aef7: Status 404 returned error can't find the container with id 7b4a68f0a4a0dbf7bec7c8c311ea7a2f0dacfe786a280d83a949d6ac5682aef7
Jul 11 23:38:17.482040 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.482002    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:38:17.482144 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.482083    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:38:17.482144 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.482126    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:38:17.691559 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.691468    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6" event={"ID":"73e09177-7cd0-493f-841d-252842535a58","Type":"ContainerStarted","Data":"9322d1b047f463db1f3f4afa0695b1df003a275619d9b9f0ee28e24d87ab6f98"}
Jul 11 23:38:17.691559 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.691538    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6" event={"ID":"73e09177-7cd0-493f-841d-252842535a58","Type":"ContainerStarted","Data":"7b4a68f0a4a0dbf7bec7c8c311ea7a2f0dacfe786a280d83a949d6ac5682aef7"}
Jul 11 23:38:17.708628 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:17.708575    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6" podStartSLOduration=0.708557443 podStartE2EDuration="708.557443ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:38:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:38:17.708202593 +0000 UTC m=+427.286295370" watchObservedRunningTime="2026-07-11 23:38:17.708557443 +0000 UTC m=+427.286650220"
Jul 11 23:38:18.343500 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:18.343461    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:18.348091 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:18.348064    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:18.695686 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:18.695606    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:18.696653 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:18.696635    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:38:18.727414 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:18.727385    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"]
Jul 11 23:38:18.730983 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:18.730968    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"
Jul 11 23:38:18.734492 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:18.734469    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 11 23:38:18.734609 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:18.734511    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kk9jd\""
Jul 11 23:38:18.752050 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:18.752016    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"]
Jul 11 23:38:18.776572 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:18.776539    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7733f6b8-9a4a-4707-908e-b1db523394f7-config-file\") pod \"limitador-limitador-64c94fd64f-2t7dc\" (UID: \"7733f6b8-9a4a-4707-908e-b1db523394f7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"
Jul 11 23:38:18.776776 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:18.776635    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsh75\" (UniqueName: \"kubernetes.io/projected/7733f6b8-9a4a-4707-908e-b1db523394f7-kube-api-access-tsh75\") pod \"limitador-limitador-64c94fd64f-2t7dc\" (UID: \"7733f6b8-9a4a-4707-908e-b1db523394f7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"
Jul 11 23:38:18.808126 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:18.808094    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"]
Jul 11 23:38:18.877183 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:18.877131    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsh75\" (UniqueName: \"kubernetes.io/projected/7733f6b8-9a4a-4707-908e-b1db523394f7-kube-api-access-tsh75\") pod \"limitador-limitador-64c94fd64f-2t7dc\" (UID: \"7733f6b8-9a4a-4707-908e-b1db523394f7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"
Jul 11 23:38:18.877396 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:18.877225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7733f6b8-9a4a-4707-908e-b1db523394f7-config-file\") pod \"limitador-limitador-64c94fd64f-2t7dc\" (UID: \"7733f6b8-9a4a-4707-908e-b1db523394f7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"
Jul 11 23:38:18.877908 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:18.877890    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7733f6b8-9a4a-4707-908e-b1db523394f7-config-file\") pod \"limitador-limitador-64c94fd64f-2t7dc\" (UID: \"7733f6b8-9a4a-4707-908e-b1db523394f7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"
Jul 11 23:38:18.885037 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:18.885016    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsh75\" (UniqueName: \"kubernetes.io/projected/7733f6b8-9a4a-4707-908e-b1db523394f7-kube-api-access-tsh75\") pod \"limitador-limitador-64c94fd64f-2t7dc\" (UID: \"7733f6b8-9a4a-4707-908e-b1db523394f7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"
Jul 11 23:38:19.042267 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:19.042237    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"
Jul 11 23:38:19.174317 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:19.174272    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"]
Jul 11 23:38:19.176335 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:38:19.176309    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7733f6b8_9a4a_4707_908e_b1db523394f7.slice/crio-ca4213d76cb2ee144f3609f8891d2d1ccd17038d4408a964f24e2f1e39498243 WatchSource:0}: Error finding container ca4213d76cb2ee144f3609f8891d2d1ccd17038d4408a964f24e2f1e39498243: Status 404 returned error can't find the container with id ca4213d76cb2ee144f3609f8891d2d1ccd17038d4408a964f24e2f1e39498243
Jul 11 23:38:19.700649 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:19.700616    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-2t7dc" event={"ID":"7733f6b8-9a4a-4707-908e-b1db523394f7","Type":"ContainerStarted","Data":"ca4213d76cb2ee144f3609f8891d2d1ccd17038d4408a964f24e2f1e39498243"}
Jul 11 23:38:22.716408 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:22.716369    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-2t7dc" event={"ID":"7733f6b8-9a4a-4707-908e-b1db523394f7","Type":"ContainerStarted","Data":"e3a8e0a79715c0517b04058b481072e317a6877725d1cffa076d20125297c642"}
Jul 11 23:38:22.716872 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:22.716547    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"
Jul 11 23:38:22.731158 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:22.731100    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-2t7dc" podStartSLOduration=2.160739133 podStartE2EDuration="4.731079968s" totalImagesPullingTime="2.570340835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:38:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:38:22.730679908 +0000 UTC m=+432.308772686" watchObservedRunningTime="2026-07-11 23:38:22.731079968 +0000 UTC m=+432.309172745"
Jul 11 23:38:33.722332 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:33.722261    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"
Jul 11 23:38:38.289141 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:38.289105    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"]
Jul 11 23:38:38.289565 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:38.289294    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-2t7dc" podUID="7733f6b8-9a4a-4707-908e-b1db523394f7" containerName="limitador" containerID="cri-o://e3a8e0a79715c0517b04058b481072e317a6877725d1cffa076d20125297c642" gracePeriod=30
Jul 11 23:38:38.781944 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:38.781910    2569 generic.go:356] "Generic (PLEG): container finished" podID="7733f6b8-9a4a-4707-908e-b1db523394f7" containerID="e3a8e0a79715c0517b04058b481072e317a6877725d1cffa076d20125297c642" exitCode=0
Jul 11 23:38:38.782080 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:38.781940    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-2t7dc" event={"ID":"7733f6b8-9a4a-4707-908e-b1db523394f7","Type":"ContainerDied","Data":"e3a8e0a79715c0517b04058b481072e317a6877725d1cffa076d20125297c642"}
Jul 11 23:38:38.833031 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:38.833006    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"
Jul 11 23:38:38.950746 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:38.950644    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsh75\" (UniqueName: \"kubernetes.io/projected/7733f6b8-9a4a-4707-908e-b1db523394f7-kube-api-access-tsh75\") pod \"7733f6b8-9a4a-4707-908e-b1db523394f7\" (UID: \"7733f6b8-9a4a-4707-908e-b1db523394f7\") "
Jul 11 23:38:38.950746 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:38.950704    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7733f6b8-9a4a-4707-908e-b1db523394f7-config-file\") pod \"7733f6b8-9a4a-4707-908e-b1db523394f7\" (UID: \"7733f6b8-9a4a-4707-908e-b1db523394f7\") "
Jul 11 23:38:38.951087 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:38.951058    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7733f6b8-9a4a-4707-908e-b1db523394f7-config-file" (OuterVolumeSpecName: "config-file") pod "7733f6b8-9a4a-4707-908e-b1db523394f7" (UID: "7733f6b8-9a4a-4707-908e-b1db523394f7"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 23:38:38.952800 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:38.952775    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7733f6b8-9a4a-4707-908e-b1db523394f7-kube-api-access-tsh75" (OuterVolumeSpecName: "kube-api-access-tsh75") pod "7733f6b8-9a4a-4707-908e-b1db523394f7" (UID: "7733f6b8-9a4a-4707-908e-b1db523394f7"). InnerVolumeSpecName "kube-api-access-tsh75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 23:38:39.052465 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:39.052393    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsh75\" (UniqueName: \"kubernetes.io/projected/7733f6b8-9a4a-4707-908e-b1db523394f7-kube-api-access-tsh75\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:38:39.052465 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:39.052451    2569 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7733f6b8-9a4a-4707-908e-b1db523394f7-config-file\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:38:39.787515 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:39.787484    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-2t7dc" event={"ID":"7733f6b8-9a4a-4707-908e-b1db523394f7","Type":"ContainerDied","Data":"ca4213d76cb2ee144f3609f8891d2d1ccd17038d4408a964f24e2f1e39498243"}
Jul 11 23:38:39.787515 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:39.787507    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"
Jul 11 23:38:39.788017 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:39.787542    2569 scope.go:117] "RemoveContainer" containerID="e3a8e0a79715c0517b04058b481072e317a6877725d1cffa076d20125297c642"
Jul 11 23:38:39.801854 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:39.801832    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"]
Jul 11 23:38:39.807218 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:39.807197    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-2t7dc"]
Jul 11 23:38:40.985914 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:40.985882    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7733f6b8-9a4a-4707-908e-b1db523394f7" path="/var/lib/kubelet/pods/7733f6b8-9a4a-4707-908e-b1db523394f7/volumes"
Jul 11 23:38:58.583082 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.583046    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5b4d8bbdd-w5bbc"]
Jul 11 23:38:58.583578 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.583561    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7733f6b8-9a4a-4707-908e-b1db523394f7" containerName="limitador"
Jul 11 23:38:58.583642 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.583582    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7733f6b8-9a4a-4707-908e-b1db523394f7" containerName="limitador"
Jul 11 23:38:58.583679 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.583656    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7733f6b8-9a4a-4707-908e-b1db523394f7" containerName="limitador"
Jul 11 23:38:58.592040 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.592015    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b4d8bbdd-w5bbc"
Jul 11 23:38:58.594334 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.594311    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b4d8bbdd-w5bbc"]
Jul 11 23:38:58.594643 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.594619    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 11 23:38:58.594736 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.594716    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 11 23:38:58.594791 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.594772    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7psqn\""
Jul 11 23:38:58.720738 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.720699    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0db9fc50-f0cf-45d0-a14a-67168dbbbb37-webhook-cert\") pod \"maas-controller-5b4d8bbdd-w5bbc\" (UID: \"0db9fc50-f0cf-45d0-a14a-67168dbbbb37\") " pod="opendatahub/maas-controller-5b4d8bbdd-w5bbc"
Jul 11 23:38:58.720920 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.720752    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g94sd\" (UniqueName: \"kubernetes.io/projected/0db9fc50-f0cf-45d0-a14a-67168dbbbb37-kube-api-access-g94sd\") pod \"maas-controller-5b4d8bbdd-w5bbc\" (UID: \"0db9fc50-f0cf-45d0-a14a-67168dbbbb37\") " pod="opendatahub/maas-controller-5b4d8bbdd-w5bbc"
Jul 11 23:38:58.746389 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.746356    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6657cc646c-5lzqf"]
Jul 11 23:38:58.749815 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.749797    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6657cc646c-5lzqf"
Jul 11 23:38:58.760224 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.760204    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6657cc646c-5lzqf"]
Jul 11 23:38:58.822167 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.822133    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g94sd\" (UniqueName: \"kubernetes.io/projected/0db9fc50-f0cf-45d0-a14a-67168dbbbb37-kube-api-access-g94sd\") pod \"maas-controller-5b4d8bbdd-w5bbc\" (UID: \"0db9fc50-f0cf-45d0-a14a-67168dbbbb37\") " pod="opendatahub/maas-controller-5b4d8bbdd-w5bbc"
Jul 11 23:38:58.822322 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.822224    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0db9fc50-f0cf-45d0-a14a-67168dbbbb37-webhook-cert\") pod \"maas-controller-5b4d8bbdd-w5bbc\" (UID: \"0db9fc50-f0cf-45d0-a14a-67168dbbbb37\") " pod="opendatahub/maas-controller-5b4d8bbdd-w5bbc"
Jul 11 23:38:58.824540 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.824517    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0db9fc50-f0cf-45d0-a14a-67168dbbbb37-webhook-cert\") pod \"maas-controller-5b4d8bbdd-w5bbc\" (UID: \"0db9fc50-f0cf-45d0-a14a-67168dbbbb37\") " pod="opendatahub/maas-controller-5b4d8bbdd-w5bbc"
Jul 11 23:38:58.829535 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.829511    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g94sd\" (UniqueName: \"kubernetes.io/projected/0db9fc50-f0cf-45d0-a14a-67168dbbbb37-kube-api-access-g94sd\") pod \"maas-controller-5b4d8bbdd-w5bbc\" (UID: \"0db9fc50-f0cf-45d0-a14a-67168dbbbb37\") " pod="opendatahub/maas-controller-5b4d8bbdd-w5bbc"
Jul 11 23:38:58.904163 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.904064    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b4d8bbdd-w5bbc"
Jul 11 23:38:58.922786 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.922754    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16ede5e8-dec7-4276-9506-d3651a8211de-webhook-cert\") pod \"maas-controller-6657cc646c-5lzqf\" (UID: \"16ede5e8-dec7-4276-9506-d3651a8211de\") " pod="opendatahub/maas-controller-6657cc646c-5lzqf"
Jul 11 23:38:58.922932 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:58.922794    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m86sp\" (UniqueName: \"kubernetes.io/projected/16ede5e8-dec7-4276-9506-d3651a8211de-kube-api-access-m86sp\") pod \"maas-controller-6657cc646c-5lzqf\" (UID: \"16ede5e8-dec7-4276-9506-d3651a8211de\") " pod="opendatahub/maas-controller-6657cc646c-5lzqf"
Jul 11 23:38:59.024229 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:59.024191    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16ede5e8-dec7-4276-9506-d3651a8211de-webhook-cert\") pod \"maas-controller-6657cc646c-5lzqf\" (UID: \"16ede5e8-dec7-4276-9506-d3651a8211de\") " pod="opendatahub/maas-controller-6657cc646c-5lzqf"
Jul 11 23:38:59.024229 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:59.024228    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m86sp\" (UniqueName: \"kubernetes.io/projected/16ede5e8-dec7-4276-9506-d3651a8211de-kube-api-access-m86sp\") pod \"maas-controller-6657cc646c-5lzqf\" (UID: \"16ede5e8-dec7-4276-9506-d3651a8211de\") " pod="opendatahub/maas-controller-6657cc646c-5lzqf"
Jul 11 23:38:59.026506 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:59.026483    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16ede5e8-dec7-4276-9506-d3651a8211de-webhook-cert\") pod \"maas-controller-6657cc646c-5lzqf\" (UID: \"16ede5e8-dec7-4276-9506-d3651a8211de\") " pod="opendatahub/maas-controller-6657cc646c-5lzqf"
Jul 11 23:38:59.031249 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:59.031226    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m86sp\" (UniqueName: \"kubernetes.io/projected/16ede5e8-dec7-4276-9506-d3651a8211de-kube-api-access-m86sp\") pod \"maas-controller-6657cc646c-5lzqf\" (UID: \"16ede5e8-dec7-4276-9506-d3651a8211de\") " pod="opendatahub/maas-controller-6657cc646c-5lzqf"
Jul 11 23:38:59.034753 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:59.034735    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5b4d8bbdd-w5bbc"]
Jul 11 23:38:59.041974 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:38:59.037593    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0db9fc50_f0cf_45d0_a14a_67168dbbbb37.slice/crio-a1182b39f9ecbcfff84d8bb50f59c3462ba6fe2b292d0be6eddb5c93eb75864f WatchSource:0}: Error finding container a1182b39f9ecbcfff84d8bb50f59c3462ba6fe2b292d0be6eddb5c93eb75864f: Status 404 returned error can't find the container with id a1182b39f9ecbcfff84d8bb50f59c3462ba6fe2b292d0be6eddb5c93eb75864f
Jul 11 23:38:59.060352 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:59.060321    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6657cc646c-5lzqf"
Jul 11 23:38:59.189178 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:59.189104    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6657cc646c-5lzqf"]
Jul 11 23:38:59.191912 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:38:59.191886    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16ede5e8_dec7_4276_9506_d3651a8211de.slice/crio-88287346339cfca5bb8ef223812d6f4930643cde9610e1fa262eeb80617362ba WatchSource:0}: Error finding container 88287346339cfca5bb8ef223812d6f4930643cde9610e1fa262eeb80617362ba: Status 404 returned error can't find the container with id 88287346339cfca5bb8ef223812d6f4930643cde9610e1fa262eeb80617362ba
Jul 11 23:38:59.879945 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:59.879902    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b4d8bbdd-w5bbc" event={"ID":"0db9fc50-f0cf-45d0-a14a-67168dbbbb37","Type":"ContainerStarted","Data":"a1182b39f9ecbcfff84d8bb50f59c3462ba6fe2b292d0be6eddb5c93eb75864f"}
Jul 11 23:38:59.881526 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:38:59.881496    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6657cc646c-5lzqf" event={"ID":"16ede5e8-dec7-4276-9506-d3651a8211de","Type":"ContainerStarted","Data":"88287346339cfca5bb8ef223812d6f4930643cde9610e1fa262eeb80617362ba"}
Jul 11 23:39:01.892413 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:01.892372    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b4d8bbdd-w5bbc" event={"ID":"0db9fc50-f0cf-45d0-a14a-67168dbbbb37","Type":"ContainerStarted","Data":"9f7a65841eaec84196eaee8e145ddd8437294bf519c6165453a88080c4245f54"}
Jul 11 23:39:01.892847 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:01.892458    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5b4d8bbdd-w5bbc"
Jul 11 23:39:01.894365 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:01.894334    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6657cc646c-5lzqf" event={"ID":"16ede5e8-dec7-4276-9506-d3651a8211de","Type":"ContainerStarted","Data":"8f91ff7ae9e6f269f2ed36c44863c246b024c1730a8fb3bb22a0cde0782bd83b"}
Jul 11 23:39:01.894491 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:01.894461    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6657cc646c-5lzqf"
Jul 11 23:39:01.912818 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:01.912770    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5b4d8bbdd-w5bbc" podStartSLOduration=1.24384742 podStartE2EDuration="3.91275265s" totalImagesPullingTime="2.66890523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:38:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:39:01.910806409 +0000 UTC m=+471.488899189" watchObservedRunningTime="2026-07-11 23:39:01.91275265 +0000 UTC m=+471.490845436"
Jul 11 23:39:01.929698 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:01.929629    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6657cc646c-5lzqf" podStartSLOduration=1.401110074 podStartE2EDuration="3.929605808s" totalImagesPullingTime="2.528495734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:38:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:39:01.92772206 +0000 UTC m=+471.505814837" watchObservedRunningTime="2026-07-11 23:39:01.929605808 +0000 UTC m=+471.507698587"
Jul 11 23:39:12.901212 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:12.901181    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6657cc646c-5lzqf"
Jul 11 23:39:12.901599 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:12.901256    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5b4d8bbdd-w5bbc"
Jul 11 23:39:12.956516 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:12.956481    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b4d8bbdd-w5bbc"]
Jul 11 23:39:12.956750 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:12.956699    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5b4d8bbdd-w5bbc" podUID="0db9fc50-f0cf-45d0-a14a-67168dbbbb37" containerName="manager" containerID="cri-o://9f7a65841eaec84196eaee8e145ddd8437294bf519c6165453a88080c4245f54" gracePeriod=30
Jul 11 23:39:13.200068 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:13.200042    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b4d8bbdd-w5bbc"
Jul 11 23:39:13.235979 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:13.235945    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0db9fc50-f0cf-45d0-a14a-67168dbbbb37-webhook-cert\") pod \"0db9fc50-f0cf-45d0-a14a-67168dbbbb37\" (UID: \"0db9fc50-f0cf-45d0-a14a-67168dbbbb37\") "
Jul 11 23:39:13.236154 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:13.236073    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g94sd\" (UniqueName: \"kubernetes.io/projected/0db9fc50-f0cf-45d0-a14a-67168dbbbb37-kube-api-access-g94sd\") pod \"0db9fc50-f0cf-45d0-a14a-67168dbbbb37\" (UID: \"0db9fc50-f0cf-45d0-a14a-67168dbbbb37\") "
Jul 11 23:39:13.238118 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:13.238094    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0db9fc50-f0cf-45d0-a14a-67168dbbbb37-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0db9fc50-f0cf-45d0-a14a-67168dbbbb37" (UID: "0db9fc50-f0cf-45d0-a14a-67168dbbbb37"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 23:39:13.238224 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:13.238145    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0db9fc50-f0cf-45d0-a14a-67168dbbbb37-kube-api-access-g94sd" (OuterVolumeSpecName: "kube-api-access-g94sd") pod "0db9fc50-f0cf-45d0-a14a-67168dbbbb37" (UID: "0db9fc50-f0cf-45d0-a14a-67168dbbbb37"). InnerVolumeSpecName "kube-api-access-g94sd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 23:39:13.337041 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:13.337006    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g94sd\" (UniqueName: \"kubernetes.io/projected/0db9fc50-f0cf-45d0-a14a-67168dbbbb37-kube-api-access-g94sd\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:39:13.337041 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:13.337037    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0db9fc50-f0cf-45d0-a14a-67168dbbbb37-webhook-cert\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:39:13.945826 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:13.945787    2569 generic.go:356] "Generic (PLEG): container finished" podID="0db9fc50-f0cf-45d0-a14a-67168dbbbb37" containerID="9f7a65841eaec84196eaee8e145ddd8437294bf519c6165453a88080c4245f54" exitCode=0
Jul 11 23:39:13.945826 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:13.945824    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5b4d8bbdd-w5bbc"
Jul 11 23:39:13.946269 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:13.945834    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b4d8bbdd-w5bbc" event={"ID":"0db9fc50-f0cf-45d0-a14a-67168dbbbb37","Type":"ContainerDied","Data":"9f7a65841eaec84196eaee8e145ddd8437294bf519c6165453a88080c4245f54"}
Jul 11 23:39:13.946269 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:13.945881    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5b4d8bbdd-w5bbc" event={"ID":"0db9fc50-f0cf-45d0-a14a-67168dbbbb37","Type":"ContainerDied","Data":"a1182b39f9ecbcfff84d8bb50f59c3462ba6fe2b292d0be6eddb5c93eb75864f"}
Jul 11 23:39:13.946269 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:13.945907    2569 scope.go:117] "RemoveContainer" containerID="9f7a65841eaec84196eaee8e145ddd8437294bf519c6165453a88080c4245f54"
Jul 11 23:39:13.954628 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:13.954607    2569 scope.go:117] "RemoveContainer" containerID="9f7a65841eaec84196eaee8e145ddd8437294bf519c6165453a88080c4245f54"
Jul 11 23:39:13.954890 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:39:13.954872    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f7a65841eaec84196eaee8e145ddd8437294bf519c6165453a88080c4245f54\": container with ID starting with 9f7a65841eaec84196eaee8e145ddd8437294bf519c6165453a88080c4245f54 not found: ID does not exist" containerID="9f7a65841eaec84196eaee8e145ddd8437294bf519c6165453a88080c4245f54"
Jul 11 23:39:13.954955 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:13.954903    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f7a65841eaec84196eaee8e145ddd8437294bf519c6165453a88080c4245f54"} err="failed to get container status \"9f7a65841eaec84196eaee8e145ddd8437294bf519c6165453a88080c4245f54\": rpc error: code = NotFound desc = could not find container \"9f7a65841eaec84196eaee8e145ddd8437294bf519c6165453a88080c4245f54\": container with ID starting with 9f7a65841eaec84196eaee8e145ddd8437294bf519c6165453a88080c4245f54 not found: ID does not exist"
Jul 11 23:39:13.965951 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:13.965928    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5b4d8bbdd-w5bbc"]
Jul 11 23:39:13.968969 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:13.968947    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5b4d8bbdd-w5bbc"]
Jul 11 23:39:14.983169 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:14.983126    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0db9fc50-f0cf-45d0-a14a-67168dbbbb37" path="/var/lib/kubelet/pods/0db9fc50-f0cf-45d0-a14a-67168dbbbb37/volumes"
Jul 11 23:39:34.117799 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.117766    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-858b984599-f4m7g"]
Jul 11 23:39:34.118270 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.118062    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0db9fc50-f0cf-45d0-a14a-67168dbbbb37" containerName="manager"
Jul 11 23:39:34.118270 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.118072    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0db9fc50-f0cf-45d0-a14a-67168dbbbb37" containerName="manager"
Jul 11 23:39:34.118270 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.118133    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0db9fc50-f0cf-45d0-a14a-67168dbbbb37" containerName="manager"
Jul 11 23:39:34.120970 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.120954    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-858b984599-f4m7g"
Jul 11 23:39:34.124631 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.124609    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 11 23:39:34.124755 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.124610    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6n9rc\""
Jul 11 23:39:34.124755 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.124643    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 11 23:39:34.124755 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.124611    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 11 23:39:34.130987 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.130958    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-858b984599-f4m7g"]
Jul 11 23:39:34.215150 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.215111    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zskct\" (UniqueName: \"kubernetes.io/projected/047de06e-cc71-43c0-b81a-6b56cc5d3196-kube-api-access-zskct\") pod \"maas-api-858b984599-f4m7g\" (UID: \"047de06e-cc71-43c0-b81a-6b56cc5d3196\") " pod="odh-ai-gateway-infra/maas-api-858b984599-f4m7g"
Jul 11 23:39:34.215329 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.215176    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/047de06e-cc71-43c0-b81a-6b56cc5d3196-maas-api-tls\") pod \"maas-api-858b984599-f4m7g\" (UID: \"047de06e-cc71-43c0-b81a-6b56cc5d3196\") " pod="odh-ai-gateway-infra/maas-api-858b984599-f4m7g"
Jul 11 23:39:34.215329 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.215211    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/047de06e-cc71-43c0-b81a-6b56cc5d3196-openshift-service-ca\") pod \"maas-api-858b984599-f4m7g\" (UID: \"047de06e-cc71-43c0-b81a-6b56cc5d3196\") " pod="odh-ai-gateway-infra/maas-api-858b984599-f4m7g"
Jul 11 23:39:34.315842 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.315809    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/047de06e-cc71-43c0-b81a-6b56cc5d3196-openshift-service-ca\") pod \"maas-api-858b984599-f4m7g\" (UID: \"047de06e-cc71-43c0-b81a-6b56cc5d3196\") " pod="odh-ai-gateway-infra/maas-api-858b984599-f4m7g"
Jul 11 23:39:34.316009 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.315866    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zskct\" (UniqueName: \"kubernetes.io/projected/047de06e-cc71-43c0-b81a-6b56cc5d3196-kube-api-access-zskct\") pod \"maas-api-858b984599-f4m7g\" (UID: \"047de06e-cc71-43c0-b81a-6b56cc5d3196\") " pod="odh-ai-gateway-infra/maas-api-858b984599-f4m7g"
Jul 11 23:39:34.316009 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.315903    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/047de06e-cc71-43c0-b81a-6b56cc5d3196-maas-api-tls\") pod \"maas-api-858b984599-f4m7g\" (UID: \"047de06e-cc71-43c0-b81a-6b56cc5d3196\") " pod="odh-ai-gateway-infra/maas-api-858b984599-f4m7g"
Jul 11 23:39:34.316448 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.316404    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/047de06e-cc71-43c0-b81a-6b56cc5d3196-openshift-service-ca\") pod \"maas-api-858b984599-f4m7g\" (UID: \"047de06e-cc71-43c0-b81a-6b56cc5d3196\") " pod="odh-ai-gateway-infra/maas-api-858b984599-f4m7g"
Jul 11 23:39:34.318283 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.318266    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/047de06e-cc71-43c0-b81a-6b56cc5d3196-maas-api-tls\") pod \"maas-api-858b984599-f4m7g\" (UID: \"047de06e-cc71-43c0-b81a-6b56cc5d3196\") " pod="odh-ai-gateway-infra/maas-api-858b984599-f4m7g"
Jul 11 23:39:34.327076 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.327053    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zskct\" (UniqueName: \"kubernetes.io/projected/047de06e-cc71-43c0-b81a-6b56cc5d3196-kube-api-access-zskct\") pod \"maas-api-858b984599-f4m7g\" (UID: \"047de06e-cc71-43c0-b81a-6b56cc5d3196\") " pod="odh-ai-gateway-infra/maas-api-858b984599-f4m7g"
Jul 11 23:39:34.432100 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.432027    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-858b984599-f4m7g"
Jul 11 23:39:34.554999 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.554963    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-skksg"]
Jul 11 23:39:34.558677 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.558652    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-skksg"
Jul 11 23:39:34.561337 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.561308    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-wzs9b\""
Jul 11 23:39:34.561337 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.561335    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 11 23:39:34.567282 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.567221    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-skksg"]
Jul 11 23:39:34.572753 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.572731    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-858b984599-f4m7g"]
Jul 11 23:39:34.576326 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:39:34.576298    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod047de06e_cc71_43c0_b81a_6b56cc5d3196.slice/crio-92a5d6965fc213b523bde61314eb099929b7822dfabb220607150ced5cfe6dd0 WatchSource:0}: Error finding container 92a5d6965fc213b523bde61314eb099929b7822dfabb220607150ced5cfe6dd0: Status 404 returned error can't find the container with id 92a5d6965fc213b523bde61314eb099929b7822dfabb220607150ced5cfe6dd0
Jul 11 23:39:34.579522 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.579496    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-98m9p"]
Jul 11 23:39:34.583501 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.583483    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-98m9p"
Jul 11 23:39:34.593404 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.593382    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-98m9p"]
Jul 11 23:39:34.619120 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.619094    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72md8\" (UniqueName: \"kubernetes.io/projected/828cf44d-5351-4eda-a056-0726258dabd1-kube-api-access-72md8\") pod \"payload-pre-processing-74584bb9f8-skksg\" (UID: \"828cf44d-5351-4eda-a056-0726258dabd1\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-skksg"
Jul 11 23:39:34.619225 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.619140    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/828cf44d-5351-4eda-a056-0726258dabd1-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-skksg\" (UID: \"828cf44d-5351-4eda-a056-0726258dabd1\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-skksg"
Jul 11 23:39:34.719916 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.719834    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72md8\" (UniqueName: \"kubernetes.io/projected/828cf44d-5351-4eda-a056-0726258dabd1-kube-api-access-72md8\") pod \"payload-pre-processing-74584bb9f8-skksg\" (UID: \"828cf44d-5351-4eda-a056-0726258dabd1\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-skksg"
Jul 11 23:39:34.719916 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.719888    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/828cf44d-5351-4eda-a056-0726258dabd1-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-skksg\" (UID: \"828cf44d-5351-4eda-a056-0726258dabd1\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-skksg"
Jul 11 23:39:34.720106 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.719925    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b44e59d-5f12-470d-8b4a-1e3183c2f8f1-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-98m9p\" (UID: \"8b44e59d-5f12-470d-8b4a-1e3183c2f8f1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-98m9p"
Jul 11 23:39:34.720106 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.719940    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhw75\" (UniqueName: \"kubernetes.io/projected/8b44e59d-5f12-470d-8b4a-1e3183c2f8f1-kube-api-access-hhw75\") pod \"payload-processing-f9b8d9b9b-98m9p\" (UID: \"8b44e59d-5f12-470d-8b4a-1e3183c2f8f1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-98m9p"
Jul 11 23:39:34.720828 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.720807    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/828cf44d-5351-4eda-a056-0726258dabd1-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-skksg\" (UID: \"828cf44d-5351-4eda-a056-0726258dabd1\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-skksg"
Jul 11 23:39:34.730774 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.730752    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72md8\" (UniqueName: \"kubernetes.io/projected/828cf44d-5351-4eda-a056-0726258dabd1-kube-api-access-72md8\") pod \"payload-pre-processing-74584bb9f8-skksg\" (UID: \"828cf44d-5351-4eda-a056-0726258dabd1\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-skksg"
Jul 11 23:39:34.821160 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.821122    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b44e59d-5f12-470d-8b4a-1e3183c2f8f1-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-98m9p\" (UID: \"8b44e59d-5f12-470d-8b4a-1e3183c2f8f1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-98m9p"
Jul 11 23:39:34.821160 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.821158    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhw75\" (UniqueName: \"kubernetes.io/projected/8b44e59d-5f12-470d-8b4a-1e3183c2f8f1-kube-api-access-hhw75\") pod \"payload-processing-f9b8d9b9b-98m9p\" (UID: \"8b44e59d-5f12-470d-8b4a-1e3183c2f8f1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-98m9p"
Jul 11 23:39:34.821944 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.821905    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b44e59d-5f12-470d-8b4a-1e3183c2f8f1-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-98m9p\" (UID: \"8b44e59d-5f12-470d-8b4a-1e3183c2f8f1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-98m9p"
Jul 11 23:39:34.828915 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.828880    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhw75\" (UniqueName: \"kubernetes.io/projected/8b44e59d-5f12-470d-8b4a-1e3183c2f8f1-kube-api-access-hhw75\") pod \"payload-processing-f9b8d9b9b-98m9p\" (UID: \"8b44e59d-5f12-470d-8b4a-1e3183c2f8f1\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-98m9p"
Jul 11 23:39:34.870622 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.870596    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-skksg"
Jul 11 23:39:34.894780 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:34.894755    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-98m9p"
Jul 11 23:39:35.033800 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:35.033768    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-858b984599-f4m7g" event={"ID":"047de06e-cc71-43c0-b81a-6b56cc5d3196","Type":"ContainerStarted","Data":"92a5d6965fc213b523bde61314eb099929b7822dfabb220607150ced5cfe6dd0"}
Jul 11 23:39:35.051654 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:35.051627    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-skksg"]
Jul 11 23:39:35.053778 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:39:35.053756    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod828cf44d_5351_4eda_a056_0726258dabd1.slice/crio-a05cc9762dd8eed674159f4ed0477ac194f9ee45495feb7e12bb49b6d7ead9f7 WatchSource:0}: Error finding container a05cc9762dd8eed674159f4ed0477ac194f9ee45495feb7e12bb49b6d7ead9f7: Status 404 returned error can't find the container with id a05cc9762dd8eed674159f4ed0477ac194f9ee45495feb7e12bb49b6d7ead9f7
Jul 11 23:39:35.079420 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:35.079397    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-98m9p"]
Jul 11 23:39:35.081581 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:39:35.081554    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b44e59d_5f12_470d_8b4a_1e3183c2f8f1.slice/crio-a4637450c206c8b471cb2c4eeb74848ee25855dcc339b324799c05a13648a8a7 WatchSource:0}: Error finding container a4637450c206c8b471cb2c4eeb74848ee25855dcc339b324799c05a13648a8a7: Status 404 returned error can't find the container with id a4637450c206c8b471cb2c4eeb74848ee25855dcc339b324799c05a13648a8a7
Jul 11 23:39:36.044465 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:36.043815    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-98m9p" event={"ID":"8b44e59d-5f12-470d-8b4a-1e3183c2f8f1","Type":"ContainerStarted","Data":"a4637450c206c8b471cb2c4eeb74848ee25855dcc339b324799c05a13648a8a7"}
Jul 11 23:39:36.046127 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:36.046092    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-skksg" event={"ID":"828cf44d-5351-4eda-a056-0726258dabd1","Type":"ContainerStarted","Data":"a05cc9762dd8eed674159f4ed0477ac194f9ee45495feb7e12bb49b6d7ead9f7"}
Jul 11 23:39:37.051352 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:37.051307    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-skksg" event={"ID":"828cf44d-5351-4eda-a056-0726258dabd1","Type":"ContainerStarted","Data":"8b27d6331c73ae29eea2e4ad722d0a63c212fa5fa7e492dd1ce73eefff5d42f6"}
Jul 11 23:39:37.051799 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:37.051460    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-skksg"
Jul 11 23:39:37.052739 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:37.052715    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-98m9p" event={"ID":"8b44e59d-5f12-470d-8b4a-1e3183c2f8f1","Type":"ContainerStarted","Data":"02ee4bcecbc51dbcc40fc58753e09f1aa202fdbbfa2299a150fc5353b77fd27b"}
Jul 11 23:39:37.052855 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:37.052763    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-98m9p"
Jul 11 23:39:37.054139 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:37.054118    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-858b984599-f4m7g" event={"ID":"047de06e-cc71-43c0-b81a-6b56cc5d3196","Type":"ContainerStarted","Data":"d51b57aa727a136c24152b858d5c201bfe65bd1fbe9918bcfbd6e9f8b1b1f613"}
Jul 11 23:39:37.054239 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:37.054232    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-858b984599-f4m7g"
Jul 11 23:39:37.083476 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:37.083398    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-98m9p" podStartSLOduration=1.452641441 podStartE2EDuration="3.083379361s" totalImagesPullingTime="1.63073792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:39:37.083081003 +0000 UTC m=+506.661173780" watchObservedRunningTime="2026-07-11 23:39:37.083379361 +0000 UTC m=+506.661472140"
Jul 11 23:39:37.084206 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:37.084158    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-skksg" podStartSLOduration=1.422586061 podStartE2EDuration="3.084146956s" totalImagesPullingTime="1.661560895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:39:37.069288572 +0000 UTC m=+506.647381350" watchObservedRunningTime="2026-07-11 23:39:37.084146956 +0000 UTC m=+506.662239733"
Jul 11 23:39:37.100010 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:37.099960    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-858b984599-f4m7g" podStartSLOduration=0.964107901 podStartE2EDuration="3.099945082s" totalImagesPullingTime="2.135837181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:39:37.098074836 +0000 UTC m=+506.676167606" watchObservedRunningTime="2026-07-11 23:39:37.099945082 +0000 UTC m=+506.678037858"
Jul 11 23:39:42.083702 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:42.083668    2569 generic.go:356] "Generic (PLEG): container finished" podID="73e09177-7cd0-493f-841d-252842535a58" containerID="9322d1b047f463db1f3f4afa0695b1df003a275619d9b9f0ee28e24d87ab6f98" exitCode=0
Jul 11 23:39:42.084072 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:42.083741    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6" event={"ID":"73e09177-7cd0-493f-841d-252842535a58","Type":"ContainerDied","Data":"9322d1b047f463db1f3f4afa0695b1df003a275619d9b9f0ee28e24d87ab6f98"}
Jul 11 23:39:42.084143 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:42.084128    2569 scope.go:117] "RemoveContainer" containerID="9322d1b047f463db1f3f4afa0695b1df003a275619d9b9f0ee28e24d87ab6f98"
Jul 11 23:39:42.085074 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:42.085047    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:39:42.085135 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:42.085119    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:39:42.085174 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:42.085150    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:39:43.074761 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:43.074723    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-858b984599-f4m7g"
Jul 11 23:39:43.092873 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:43.092839    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6" event={"ID":"73e09177-7cd0-493f-841d-252842535a58","Type":"ContainerStarted","Data":"c7a39b673c36266b25a80f901a2512c228712c4fa00e76d2fe7da312a500610b"}
Jul 11 23:39:43.343294 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:43.343200    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:39:43.343512 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:43.343487    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hz4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Jul 11 23:39:43.343597 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:43.343543    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6" podUID="73e09177-7cd0-493f-841d-252842535a58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Jul 11 23:39:44.343015 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:44.342976    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hz4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Jul 11 23:39:44.343408 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:44.343045    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6" podUID="73e09177-7cd0-493f-841d-252842535a58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Jul 11 23:39:45.343369 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:45.343332    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hz4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Jul 11 23:39:45.343772 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:45.343407    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6" podUID="73e09177-7cd0-493f-841d-252842535a58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Jul 11 23:39:46.343129 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:46.343088    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-hz4g6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body=
Jul 11 23:39:46.343304 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:46.343143    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6" podUID="73e09177-7cd0-493f-841d-252842535a58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused"
Jul 11 23:39:47.346677 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:47.346650    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:39:47.347052 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:47.346973    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:39:47.347701 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:47.347681    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-hz4g6"
Jul 11 23:39:48.060563 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:48.060522    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-98m9p"
Jul 11 23:39:48.060929 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:48.060912    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-skksg"
Jul 11 23:39:53.204406 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.204366    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"]
Jul 11 23:39:53.207533 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.207510    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.210120 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.210094    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kq42x\""
Jul 11 23:39:53.210247 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.210109    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 11 23:39:53.210247 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.210193    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 11 23:39:53.211154 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.211135    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 11 23:39:53.215755 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.215691    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"]
Jul 11 23:39:53.296870 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.296833    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d3bc893-8670-47d0-aff2-2e8d6b92a68e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5\" (UID: \"3d3bc893-8670-47d0-aff2-2e8d6b92a68e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.297017 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.296894    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d3bc893-8670-47d0-aff2-2e8d6b92a68e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5\" (UID: \"3d3bc893-8670-47d0-aff2-2e8d6b92a68e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.297017 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.296947    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d3bc893-8670-47d0-aff2-2e8d6b92a68e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5\" (UID: \"3d3bc893-8670-47d0-aff2-2e8d6b92a68e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.297017 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.296998    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d3bc893-8670-47d0-aff2-2e8d6b92a68e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5\" (UID: \"3d3bc893-8670-47d0-aff2-2e8d6b92a68e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.297136 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.297024    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5hwn\" (UniqueName: \"kubernetes.io/projected/3d3bc893-8670-47d0-aff2-2e8d6b92a68e-kube-api-access-g5hwn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5\" (UID: \"3d3bc893-8670-47d0-aff2-2e8d6b92a68e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.297136 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.297044    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d3bc893-8670-47d0-aff2-2e8d6b92a68e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5\" (UID: \"3d3bc893-8670-47d0-aff2-2e8d6b92a68e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.397888 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.397848    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d3bc893-8670-47d0-aff2-2e8d6b92a68e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5\" (UID: \"3d3bc893-8670-47d0-aff2-2e8d6b92a68e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.398071 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.397908    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d3bc893-8670-47d0-aff2-2e8d6b92a68e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5\" (UID: \"3d3bc893-8670-47d0-aff2-2e8d6b92a68e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.398071 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.397930    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d3bc893-8670-47d0-aff2-2e8d6b92a68e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5\" (UID: \"3d3bc893-8670-47d0-aff2-2e8d6b92a68e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.398146 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.398055    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d3bc893-8670-47d0-aff2-2e8d6b92a68e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5\" (UID: \"3d3bc893-8670-47d0-aff2-2e8d6b92a68e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.398146 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.398117    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5hwn\" (UniqueName: \"kubernetes.io/projected/3d3bc893-8670-47d0-aff2-2e8d6b92a68e-kube-api-access-g5hwn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5\" (UID: \"3d3bc893-8670-47d0-aff2-2e8d6b92a68e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.398208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.398146    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d3bc893-8670-47d0-aff2-2e8d6b92a68e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5\" (UID: \"3d3bc893-8670-47d0-aff2-2e8d6b92a68e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.398324 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.398306    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d3bc893-8670-47d0-aff2-2e8d6b92a68e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5\" (UID: \"3d3bc893-8670-47d0-aff2-2e8d6b92a68e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.398387 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.398359    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d3bc893-8670-47d0-aff2-2e8d6b92a68e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5\" (UID: \"3d3bc893-8670-47d0-aff2-2e8d6b92a68e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.398551 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.398460    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d3bc893-8670-47d0-aff2-2e8d6b92a68e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5\" (UID: \"3d3bc893-8670-47d0-aff2-2e8d6b92a68e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.400325 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.400309    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d3bc893-8670-47d0-aff2-2e8d6b92a68e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5\" (UID: \"3d3bc893-8670-47d0-aff2-2e8d6b92a68e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.400456 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.400420    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d3bc893-8670-47d0-aff2-2e8d6b92a68e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5\" (UID: \"3d3bc893-8670-47d0-aff2-2e8d6b92a68e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.405263 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.405239    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5hwn\" (UniqueName: \"kubernetes.io/projected/3d3bc893-8670-47d0-aff2-2e8d6b92a68e-kube-api-access-g5hwn\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5\" (UID: \"3d3bc893-8670-47d0-aff2-2e8d6b92a68e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.519328 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.519232    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:39:53.648187 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:53.648155    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"]
Jul 11 23:39:53.651094 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:39:53.651066    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d3bc893_8670_47d0_aff2_2e8d6b92a68e.slice/crio-3807f697ee00dcd6f6d21aec621807f4558921e240205e972a97d68eaa127cca WatchSource:0}: Error finding container 3807f697ee00dcd6f6d21aec621807f4558921e240205e972a97d68eaa127cca: Status 404 returned error can't find the container with id 3807f697ee00dcd6f6d21aec621807f4558921e240205e972a97d68eaa127cca
Jul 11 23:39:54.143088 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:54.143043    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5" event={"ID":"3d3bc893-8670-47d0-aff2-2e8d6b92a68e","Type":"ContainerStarted","Data":"3807f697ee00dcd6f6d21aec621807f4558921e240205e972a97d68eaa127cca"}
Jul 11 23:39:55.148657 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:55.148559    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5" event={"ID":"3d3bc893-8670-47d0-aff2-2e8d6b92a68e","Type":"ContainerStarted","Data":"e78a8e2cd8e3150485de96d1443f693bbacf44bf4f12d2b192e277d22a3bbce0"}
Jul 11 23:39:55.181399 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:39:55.181338    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5" podStartSLOduration=0.936505526 podStartE2EDuration="2.181318084s" totalImagesPullingTime="1.244812558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:39:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:39:55.180260189 +0000 UTC m=+524.758352966" watchObservedRunningTime="2026-07-11 23:39:55.181318084 +0000 UTC m=+524.759410863"
Jul 11 23:40:03.396135 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.396051    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"]
Jul 11 23:40:03.400009 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.399984    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.402526 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.402501    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 11 23:40:03.410865 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.410835    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"]
Jul 11 23:40:03.519531 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.519495    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:40:03.519706 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.519554    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:40:03.532506 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.532477    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:40:03.589818 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.589781    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pmw6\" (UniqueName: \"kubernetes.io/projected/f263dd7b-87c9-410b-b14e-e24a4867f10b-kube-api-access-6pmw6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pl8x\" (UID: \"f263dd7b-87c9-410b-b14e-e24a4867f10b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.589818 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.589821    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f263dd7b-87c9-410b-b14e-e24a4867f10b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pl8x\" (UID: \"f263dd7b-87c9-410b-b14e-e24a4867f10b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.590036 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.589846    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f263dd7b-87c9-410b-b14e-e24a4867f10b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pl8x\" (UID: \"f263dd7b-87c9-410b-b14e-e24a4867f10b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.590036 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.589923    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f263dd7b-87c9-410b-b14e-e24a4867f10b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pl8x\" (UID: \"f263dd7b-87c9-410b-b14e-e24a4867f10b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.590036 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.589955    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f263dd7b-87c9-410b-b14e-e24a4867f10b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pl8x\" (UID: \"f263dd7b-87c9-410b-b14e-e24a4867f10b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.590036 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.590019    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f263dd7b-87c9-410b-b14e-e24a4867f10b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pl8x\" (UID: \"f263dd7b-87c9-410b-b14e-e24a4867f10b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.691226 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.691116    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f263dd7b-87c9-410b-b14e-e24a4867f10b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pl8x\" (UID: \"f263dd7b-87c9-410b-b14e-e24a4867f10b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.691226 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.691182    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f263dd7b-87c9-410b-b14e-e24a4867f10b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pl8x\" (UID: \"f263dd7b-87c9-410b-b14e-e24a4867f10b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.691226 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.691210    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f263dd7b-87c9-410b-b14e-e24a4867f10b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pl8x\" (UID: \"f263dd7b-87c9-410b-b14e-e24a4867f10b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.691569 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.691322    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f263dd7b-87c9-410b-b14e-e24a4867f10b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pl8x\" (UID: \"f263dd7b-87c9-410b-b14e-e24a4867f10b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.691569 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.691476    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pmw6\" (UniqueName: \"kubernetes.io/projected/f263dd7b-87c9-410b-b14e-e24a4867f10b-kube-api-access-6pmw6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pl8x\" (UID: \"f263dd7b-87c9-410b-b14e-e24a4867f10b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.691569 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.691517    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f263dd7b-87c9-410b-b14e-e24a4867f10b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pl8x\" (UID: \"f263dd7b-87c9-410b-b14e-e24a4867f10b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.691727 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.691578    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f263dd7b-87c9-410b-b14e-e24a4867f10b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pl8x\" (UID: \"f263dd7b-87c9-410b-b14e-e24a4867f10b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.691727 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.691719    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f263dd7b-87c9-410b-b14e-e24a4867f10b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pl8x\" (UID: \"f263dd7b-87c9-410b-b14e-e24a4867f10b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.691838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.691819    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f263dd7b-87c9-410b-b14e-e24a4867f10b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pl8x\" (UID: \"f263dd7b-87c9-410b-b14e-e24a4867f10b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.693621 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.693596    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f263dd7b-87c9-410b-b14e-e24a4867f10b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pl8x\" (UID: \"f263dd7b-87c9-410b-b14e-e24a4867f10b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.693860 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.693846    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f263dd7b-87c9-410b-b14e-e24a4867f10b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pl8x\" (UID: \"f263dd7b-87c9-410b-b14e-e24a4867f10b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.698914 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.698888    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pmw6\" (UniqueName: \"kubernetes.io/projected/f263dd7b-87c9-410b-b14e-e24a4867f10b-kube-api-access-6pmw6\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pl8x\" (UID: \"f263dd7b-87c9-410b-b14e-e24a4867f10b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.710844 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.710818    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:03.842595 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:03.842569    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"]
Jul 11 23:40:03.844824 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:40:03.844803    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf263dd7b_87c9_410b_b14e_e24a4867f10b.slice/crio-5e490ed2ccc116113268b6cca1a171158b3757093924e77a9696bd29daddff66 WatchSource:0}: Error finding container 5e490ed2ccc116113268b6cca1a171158b3757093924e77a9696bd29daddff66: Status 404 returned error can't find the container with id 5e490ed2ccc116113268b6cca1a171158b3757093924e77a9696bd29daddff66
Jul 11 23:40:04.191764 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:04.191724    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x" event={"ID":"f263dd7b-87c9-410b-b14e-e24a4867f10b","Type":"ContainerStarted","Data":"bac44323d749aba29f768d6001c07f5f03521d86cc6cd468c4a0e88da9f71526"}
Jul 11 23:40:04.192007 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:04.191779    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x" event={"ID":"f263dd7b-87c9-410b-b14e-e24a4867f10b","Type":"ContainerStarted","Data":"5e490ed2ccc116113268b6cca1a171158b3757093924e77a9696bd29daddff66"}
Jul 11 23:40:04.203417 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:04.203388    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5"
Jul 11 23:40:04.208311 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:04.208262    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x" podStartSLOduration=1.035961776 podStartE2EDuration="1.208247311s" totalImagesPullingTime="172.285535ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:40:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:40:04.206802441 +0000 UTC m=+533.784895227" watchObservedRunningTime="2026-07-11 23:40:04.208247311 +0000 UTC m=+533.786340132"
Jul 11 23:40:13.711073 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:13.711020    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:13.711561 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:13.711117    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:13.723766 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:13.723739    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:40:14.245462 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:40:14.245435    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pl8x"
Jul 11 23:41:11.785282 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:41:11.785247    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 11 23:41:11.785821 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:41:11.785805    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 11 23:42:59.632808 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:42:59.632773    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6657cc646c-5lzqf"]
Jul 11 23:42:59.633348 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:42:59.633008    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6657cc646c-5lzqf" podUID="16ede5e8-dec7-4276-9506-d3651a8211de" containerName="manager" containerID="cri-o://8f91ff7ae9e6f269f2ed36c44863c246b024c1730a8fb3bb22a0cde0782bd83b" gracePeriod=30
Jul 11 23:42:59.882739 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:42:59.882712    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6657cc646c-5lzqf"
Jul 11 23:42:59.970095 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:42:59.970000    2569 generic.go:356] "Generic (PLEG): container finished" podID="16ede5e8-dec7-4276-9506-d3651a8211de" containerID="8f91ff7ae9e6f269f2ed36c44863c246b024c1730a8fb3bb22a0cde0782bd83b" exitCode=0
Jul 11 23:42:59.970095 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:42:59.970047    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6657cc646c-5lzqf"
Jul 11 23:42:59.970299 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:42:59.970085    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6657cc646c-5lzqf" event={"ID":"16ede5e8-dec7-4276-9506-d3651a8211de","Type":"ContainerDied","Data":"8f91ff7ae9e6f269f2ed36c44863c246b024c1730a8fb3bb22a0cde0782bd83b"}
Jul 11 23:42:59.970299 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:42:59.970142    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6657cc646c-5lzqf" event={"ID":"16ede5e8-dec7-4276-9506-d3651a8211de","Type":"ContainerDied","Data":"88287346339cfca5bb8ef223812d6f4930643cde9610e1fa262eeb80617362ba"}
Jul 11 23:42:59.970299 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:42:59.970170    2569 scope.go:117] "RemoveContainer" containerID="8f91ff7ae9e6f269f2ed36c44863c246b024c1730a8fb3bb22a0cde0782bd83b"
Jul 11 23:42:59.978348 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:42:59.978333    2569 scope.go:117] "RemoveContainer" containerID="8f91ff7ae9e6f269f2ed36c44863c246b024c1730a8fb3bb22a0cde0782bd83b"
Jul 11 23:42:59.978632 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:42:59.978615    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f91ff7ae9e6f269f2ed36c44863c246b024c1730a8fb3bb22a0cde0782bd83b\": container with ID starting with 8f91ff7ae9e6f269f2ed36c44863c246b024c1730a8fb3bb22a0cde0782bd83b not found: ID does not exist" containerID="8f91ff7ae9e6f269f2ed36c44863c246b024c1730a8fb3bb22a0cde0782bd83b"
Jul 11 23:42:59.978701 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:42:59.978644    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f91ff7ae9e6f269f2ed36c44863c246b024c1730a8fb3bb22a0cde0782bd83b"} err="failed to get container status \"8f91ff7ae9e6f269f2ed36c44863c246b024c1730a8fb3bb22a0cde0782bd83b\": rpc error: code = NotFound desc = could not find container \"8f91ff7ae9e6f269f2ed36c44863c246b024c1730a8fb3bb22a0cde0782bd83b\": container with ID starting with 8f91ff7ae9e6f269f2ed36c44863c246b024c1730a8fb3bb22a0cde0782bd83b not found: ID does not exist"
Jul 11 23:43:00.019973 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:00.019937    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16ede5e8-dec7-4276-9506-d3651a8211de-webhook-cert\") pod \"16ede5e8-dec7-4276-9506-d3651a8211de\" (UID: \"16ede5e8-dec7-4276-9506-d3651a8211de\") "
Jul 11 23:43:00.020158 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:00.020051    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m86sp\" (UniqueName: \"kubernetes.io/projected/16ede5e8-dec7-4276-9506-d3651a8211de-kube-api-access-m86sp\") pod \"16ede5e8-dec7-4276-9506-d3651a8211de\" (UID: \"16ede5e8-dec7-4276-9506-d3651a8211de\") "
Jul 11 23:43:00.022095 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:00.022060    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16ede5e8-dec7-4276-9506-d3651a8211de-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "16ede5e8-dec7-4276-9506-d3651a8211de" (UID: "16ede5e8-dec7-4276-9506-d3651a8211de"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 23:43:00.022214 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:00.022129    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16ede5e8-dec7-4276-9506-d3651a8211de-kube-api-access-m86sp" (OuterVolumeSpecName: "kube-api-access-m86sp") pod "16ede5e8-dec7-4276-9506-d3651a8211de" (UID: "16ede5e8-dec7-4276-9506-d3651a8211de"). InnerVolumeSpecName "kube-api-access-m86sp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 23:43:00.121726 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:00.121686    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16ede5e8-dec7-4276-9506-d3651a8211de-webhook-cert\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:43:00.121726 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:00.121718    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m86sp\" (UniqueName: \"kubernetes.io/projected/16ede5e8-dec7-4276-9506-d3651a8211de-kube-api-access-m86sp\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:43:00.290471 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:00.290440    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6657cc646c-5lzqf"]
Jul 11 23:43:00.293992 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:00.293965    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6657cc646c-5lzqf"]
Jul 11 23:43:00.981838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:00.981806    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16ede5e8-dec7-4276-9506-d3651a8211de" path="/var/lib/kubelet/pods/16ede5e8-dec7-4276-9506-d3651a8211de/volumes"
Jul 11 23:43:01.038602 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.038571    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6657cc646c-k25f5"]
Jul 11 23:43:01.038896 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.038885    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16ede5e8-dec7-4276-9506-d3651a8211de" containerName="manager"
Jul 11 23:43:01.038937 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.038898    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="16ede5e8-dec7-4276-9506-d3651a8211de" containerName="manager"
Jul 11 23:43:01.038975 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.038954    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="16ede5e8-dec7-4276-9506-d3651a8211de" containerName="manager"
Jul 11 23:43:01.043082 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.043062    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6657cc646c-k25f5"
Jul 11 23:43:01.046640 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.046620    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7psqn\""
Jul 11 23:43:01.046640 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.046637    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 11 23:43:01.046784 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.046636    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 11 23:43:01.050215 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.050189    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6657cc646c-k25f5"]
Jul 11 23:43:01.130355 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.130321    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fwp7\" (UniqueName: \"kubernetes.io/projected/11d98225-9974-407e-b717-b818b2d14f99-kube-api-access-9fwp7\") pod \"maas-controller-6657cc646c-k25f5\" (UID: \"11d98225-9974-407e-b717-b818b2d14f99\") " pod="opendatahub/maas-controller-6657cc646c-k25f5"
Jul 11 23:43:01.130573 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.130478    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11d98225-9974-407e-b717-b818b2d14f99-webhook-cert\") pod \"maas-controller-6657cc646c-k25f5\" (UID: \"11d98225-9974-407e-b717-b818b2d14f99\") " pod="opendatahub/maas-controller-6657cc646c-k25f5"
Jul 11 23:43:01.231383 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.231351    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11d98225-9974-407e-b717-b818b2d14f99-webhook-cert\") pod \"maas-controller-6657cc646c-k25f5\" (UID: \"11d98225-9974-407e-b717-b818b2d14f99\") " pod="opendatahub/maas-controller-6657cc646c-k25f5"
Jul 11 23:43:01.231609 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.231416    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fwp7\" (UniqueName: \"kubernetes.io/projected/11d98225-9974-407e-b717-b818b2d14f99-kube-api-access-9fwp7\") pod \"maas-controller-6657cc646c-k25f5\" (UID: \"11d98225-9974-407e-b717-b818b2d14f99\") " pod="opendatahub/maas-controller-6657cc646c-k25f5"
Jul 11 23:43:01.233848 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.233794    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/11d98225-9974-407e-b717-b818b2d14f99-webhook-cert\") pod \"maas-controller-6657cc646c-k25f5\" (UID: \"11d98225-9974-407e-b717-b818b2d14f99\") " pod="opendatahub/maas-controller-6657cc646c-k25f5"
Jul 11 23:43:01.239127 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.239092    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fwp7\" (UniqueName: \"kubernetes.io/projected/11d98225-9974-407e-b717-b818b2d14f99-kube-api-access-9fwp7\") pod \"maas-controller-6657cc646c-k25f5\" (UID: \"11d98225-9974-407e-b717-b818b2d14f99\") " pod="opendatahub/maas-controller-6657cc646c-k25f5"
Jul 11 23:43:01.354957 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.354905    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6657cc646c-k25f5"
Jul 11 23:43:01.484317 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.484218    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6657cc646c-k25f5"]
Jul 11 23:43:01.486758 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:43:01.486732    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d98225_9974_407e_b717_b818b2d14f99.slice/crio-4b43bc6a5c4709d3cf40d5ec16f3dfca57bf255987efda98b127d8b73fc71dd5 WatchSource:0}: Error finding container 4b43bc6a5c4709d3cf40d5ec16f3dfca57bf255987efda98b127d8b73fc71dd5: Status 404 returned error can't find the container with id 4b43bc6a5c4709d3cf40d5ec16f3dfca57bf255987efda98b127d8b73fc71dd5
Jul 11 23:43:01.488088 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.488069    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 11 23:43:01.982092 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.982057    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6657cc646c-k25f5" event={"ID":"11d98225-9974-407e-b717-b818b2d14f99","Type":"ContainerStarted","Data":"23682e640a9f3da9f3b599b4c0f51fb3d9fe2749a3451c503b808552a685fcc0"}
Jul 11 23:43:01.982483 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.982121    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6657cc646c-k25f5" event={"ID":"11d98225-9974-407e-b717-b818b2d14f99","Type":"ContainerStarted","Data":"4b43bc6a5c4709d3cf40d5ec16f3dfca57bf255987efda98b127d8b73fc71dd5"}
Jul 11 23:43:01.982483 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.982151    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6657cc646c-k25f5"
Jul 11 23:43:01.997668 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:01.997618    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6657cc646c-k25f5" podStartSLOduration=0.557069976 podStartE2EDuration="997.599692ms" totalImagesPullingTime="440.529716ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:43:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:43:01.996617312 +0000 UTC m=+711.574710090" watchObservedRunningTime="2026-07-11 23:43:01.997599692 +0000 UTC m=+711.575692506"
Jul 11 23:43:12.989611 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:43:12.989578    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6657cc646c-k25f5"
Jul 11 23:45:00.142956 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:45:00.142910    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29730225-nn99c"]
Jul 11 23:45:00.146285 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:45:00.146258    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730225-nn99c"
Jul 11 23:45:00.148814 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:45:00.148796    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\""
Jul 11 23:45:00.155556 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:45:00.155539    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730225-nn99c"
Jul 11 23:45:00.162218 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:45:00.162194    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29730225-nn99c"]
Jul 11 23:45:00.284220 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:45:00.284191    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29730225-nn99c"]
Jul 11 23:45:00.286638 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:45:00.286610    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceb9e99e_7f24_4f94_8f3c_a17214f8f00d.slice/crio-d351aba6bffe7bb61cf8f701e0287dbbc1e5abfa0ee2187e6666914b780a249c WatchSource:0}: Error finding container d351aba6bffe7bb61cf8f701e0287dbbc1e5abfa0ee2187e6666914b780a249c: Status 404 returned error can't find the container with id d351aba6bffe7bb61cf8f701e0287dbbc1e5abfa0ee2187e6666914b780a249c
Jul 11 23:45:00.504545 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:45:00.504462    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730225-nn99c" event={"ID":"ceb9e99e-7f24-4f94-8f3c-a17214f8f00d","Type":"ContainerStarted","Data":"d351aba6bffe7bb61cf8f701e0287dbbc1e5abfa0ee2187e6666914b780a249c"}
Jul 11 23:45:03.518851 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:45:03.518813    2569 generic.go:356] "Generic (PLEG): container finished" podID="ceb9e99e-7f24-4f94-8f3c-a17214f8f00d" containerID="1d4dbd82d39cc6758d5c6ba10e39feb9dfee4880e3c5f1fd8f7ecc1e73762bdc" exitCode=0
Jul 11 23:45:03.519232 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:45:03.518882    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730225-nn99c" event={"ID":"ceb9e99e-7f24-4f94-8f3c-a17214f8f00d","Type":"ContainerDied","Data":"1d4dbd82d39cc6758d5c6ba10e39feb9dfee4880e3c5f1fd8f7ecc1e73762bdc"}
Jul 11 23:45:04.659830 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:45:04.659798    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730225-nn99c"
Jul 11 23:45:05.529295 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:45:05.529271    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730225-nn99c"
Jul 11 23:45:05.529508 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:45:05.529264    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730225-nn99c" event={"ID":"ceb9e99e-7f24-4f94-8f3c-a17214f8f00d","Type":"ContainerDied","Data":"d351aba6bffe7bb61cf8f701e0287dbbc1e5abfa0ee2187e6666914b780a249c"}
Jul 11 23:45:05.529508 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:45:05.529406    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d351aba6bffe7bb61cf8f701e0287dbbc1e5abfa0ee2187e6666914b780a249c"
Jul 11 23:46:11.810779 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:46:11.810749    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 11 23:46:11.811208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:46:11.811143    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 11 23:51:11.836066 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:51:11.836037    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 11 23:51:11.838948 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:51:11.838926    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 11 23:55:06.261354 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.261236    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"]
Jul 11 23:55:06.261952 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.261783    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceb9e99e-7f24-4f94-8f3c-a17214f8f00d" containerName="cleanup"
Jul 11 23:55:06.261952 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.261802    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceb9e99e-7f24-4f94-8f3c-a17214f8f00d" containerName="cleanup"
Jul 11 23:55:06.261952 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.261911    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceb9e99e-7f24-4f94-8f3c-a17214f8f00d" containerName="cleanup"
Jul 11 23:55:06.263883 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.263859    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.266556 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.266533    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-eb1c09fd-openshift-default-dockercfg-c5gr5\""
Jul 11 23:55:06.285531 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.285499    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"]
Jul 11 23:55:06.418748 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.418710    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-workload-certs\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.418748 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.418746    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-data\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.418955 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.418774    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-podinfo\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.418955 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.418846    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-token\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.418955 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.418881    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7d2z\" (UniqueName: \"kubernetes.io/projected/0e9550d2-3722-42d3-97dd-3477c52ae66f-kube-api-access-w7d2z\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.418955 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.418912    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-credential-socket\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.418955 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.418933    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-envoy\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.418955 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.418953    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-workload-socket\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.419142 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.418969    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e9550d2-3722-42d3-97dd-3477c52ae66f-istiod-ca-cert\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.519637 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.519540    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-workload-certs\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.519637 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.519578    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-data\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.519873 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.519702    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-podinfo\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.519873 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.519763    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-token\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.519873 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.519784    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7d2z\" (UniqueName: \"kubernetes.io/projected/0e9550d2-3722-42d3-97dd-3477c52ae66f-kube-api-access-w7d2z\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.519873 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.519815    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-credential-socket\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.519873 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.519831    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-envoy\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.520131 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.519974    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-data\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.520131 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.520000    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-workload-certs\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.520131 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.520065    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-workload-socket\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.520284 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.520139    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e9550d2-3722-42d3-97dd-3477c52ae66f-istiod-ca-cert\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.520284 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.520166    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-credential-socket\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.520401 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.520343    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-workload-socket\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.520760 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.520740    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e9550d2-3722-42d3-97dd-3477c52ae66f-istiod-ca-cert\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.522004 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.521978    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-envoy\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.522107 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.522089    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-podinfo\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.530838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.530801    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-token\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.530970 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.530869    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7d2z\" (UniqueName: \"kubernetes.io/projected/0e9550d2-3722-42d3-97dd-3477c52ae66f-kube-api-access-w7d2z\") pod \"e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") " pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.574998 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.574960    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:06.708261 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.708232    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"]
Jul 11 23:55:06.710413 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:55:06.710389    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e9550d2_3722_42d3_97dd_3477c52ae66f.slice/crio-9a1b80aa0f3368c06c07bb09a1493892850ac5b3dd514452d631ee8bad78a324 WatchSource:0}: Error finding container 9a1b80aa0f3368c06c07bb09a1493892850ac5b3dd514452d631ee8bad78a324: Status 404 returned error can't find the container with id 9a1b80aa0f3368c06c07bb09a1493892850ac5b3dd514452d631ee8bad78a324
Jul 11 23:55:06.712596 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.712562    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:55:06.712666 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.712635    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:55:06.712707 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:06.712666    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:55:07.170564 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:07.170529    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6" event={"ID":"0e9550d2-3722-42d3-97dd-3477c52ae66f","Type":"ContainerStarted","Data":"8348920b86a5fd54f9e0cb6d2bb9934ee7693936f946688b58c70edd88600aa8"}
Jul 11 23:55:07.170735 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:07.170582    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6" event={"ID":"0e9550d2-3722-42d3-97dd-3477c52ae66f","Type":"ContainerStarted","Data":"9a1b80aa0f3368c06c07bb09a1493892850ac5b3dd514452d631ee8bad78a324"}
Jul 11 23:55:07.189418 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:07.189354    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6" podStartSLOduration=1.189332148 podStartE2EDuration="1.189332148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:55:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:55:07.18872187 +0000 UTC m=+1436.766814648" watchObservedRunningTime="2026-07-11 23:55:07.189332148 +0000 UTC m=+1436.767424926"
Jul 11 23:55:07.475639 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:07.475545    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"]
Jul 11 23:55:07.477806 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:07.477787    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"
Jul 11 23:55:07.480346 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:07.480322    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-eb1c09fd-serving-cert\""
Jul 11 23:55:07.493065 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:07.493028    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"]
Jul 11 23:55:07.575436 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:07.575401    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:07.590608 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:07.590564    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:07.632608 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:07.632569    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff0e692a-67bc-474f-ae49-09cc05d2be67-openshift-service-ca\") pod \"maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5\" (UID: \"ff0e692a-67bc-474f-ae49-09cc05d2be67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"
Jul 11 23:55:07.632805 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:07.632630    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgn6v\" (UniqueName: \"kubernetes.io/projected/ff0e692a-67bc-474f-ae49-09cc05d2be67-kube-api-access-xgn6v\") pod \"maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5\" (UID: \"ff0e692a-67bc-474f-ae49-09cc05d2be67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"
Jul 11 23:55:07.632805 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:07.632779    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff0e692a-67bc-474f-ae49-09cc05d2be67-maas-api-tls\") pod \"maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5\" (UID: \"ff0e692a-67bc-474f-ae49-09cc05d2be67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"
Jul 11 23:55:07.733842 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:07.733756    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgn6v\" (UniqueName: \"kubernetes.io/projected/ff0e692a-67bc-474f-ae49-09cc05d2be67-kube-api-access-xgn6v\") pod \"maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5\" (UID: \"ff0e692a-67bc-474f-ae49-09cc05d2be67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"
Jul 11 23:55:07.733986 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:07.733847    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff0e692a-67bc-474f-ae49-09cc05d2be67-maas-api-tls\") pod \"maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5\" (UID: \"ff0e692a-67bc-474f-ae49-09cc05d2be67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"
Jul 11 23:55:07.733986 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:07.733906    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff0e692a-67bc-474f-ae49-09cc05d2be67-openshift-service-ca\") pod \"maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5\" (UID: \"ff0e692a-67bc-474f-ae49-09cc05d2be67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"
Jul 11 23:55:07.734055 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:55:07.733999    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-serving-cert: secret "maas-api-e2e-ait-eb1c09fd-serving-cert" not found
Jul 11 23:55:07.734088 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:55:07.734071    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff0e692a-67bc-474f-ae49-09cc05d2be67-maas-api-tls podName:ff0e692a-67bc-474f-ae49-09cc05d2be67 nodeName:}" failed. No retries permitted until 2026-07-11 23:55:08.234053645 +0000 UTC m=+1437.812146409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ff0e692a-67bc-474f-ae49-09cc05d2be67-maas-api-tls") pod "maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5" (UID: "ff0e692a-67bc-474f-ae49-09cc05d2be67") : secret "maas-api-e2e-ait-eb1c09fd-serving-cert" not found
Jul 11 23:55:07.734600 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:07.734582    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff0e692a-67bc-474f-ae49-09cc05d2be67-openshift-service-ca\") pod \"maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5\" (UID: \"ff0e692a-67bc-474f-ae49-09cc05d2be67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"
Jul 11 23:55:07.756123 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:07.756087    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgn6v\" (UniqueName: \"kubernetes.io/projected/ff0e692a-67bc-474f-ae49-09cc05d2be67-kube-api-access-xgn6v\") pod \"maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5\" (UID: \"ff0e692a-67bc-474f-ae49-09cc05d2be67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"
Jul 11 23:55:08.175628 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:08.175598    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:08.176530 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:08.176512    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:08.239334 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:08.238466    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff0e692a-67bc-474f-ae49-09cc05d2be67-maas-api-tls\") pod \"maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5\" (UID: \"ff0e692a-67bc-474f-ae49-09cc05d2be67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"
Jul 11 23:55:08.246514 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:08.246461    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff0e692a-67bc-474f-ae49-09cc05d2be67-maas-api-tls\") pod \"maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5\" (UID: \"ff0e692a-67bc-474f-ae49-09cc05d2be67\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"
Jul 11 23:55:08.390649 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:08.390608    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"
Jul 11 23:55:08.523649 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:08.523625    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"]
Jul 11 23:55:08.525636 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:55:08.525603    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff0e692a_67bc_474f_ae49_09cc05d2be67.slice/crio-a482caa78e69a21aee828e261a5a6df5b4cc496ad5fc0e0e421929686bce081a WatchSource:0}: Error finding container a482caa78e69a21aee828e261a5a6df5b4cc496ad5fc0e0e421929686bce081a: Status 404 returned error can't find the container with id a482caa78e69a21aee828e261a5a6df5b4cc496ad5fc0e0e421929686bce081a
Jul 11 23:55:08.527237 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:08.527219    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 11 23:55:09.181379 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:09.181281    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5" event={"ID":"ff0e692a-67bc-474f-ae49-09cc05d2be67","Type":"ContainerStarted","Data":"66303779c42ac124be50984019438eb144af1da82644c04a27b4278275805506"}
Jul 11 23:55:09.181576 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:09.181479    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5" event={"ID":"ff0e692a-67bc-474f-ae49-09cc05d2be67","Type":"ContainerStarted","Data":"a482caa78e69a21aee828e261a5a6df5b4cc496ad5fc0e0e421929686bce081a"}
Jul 11 23:55:09.203332 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:09.203273    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5" podStartSLOduration=1.837484791 podStartE2EDuration="2.203256915s" totalImagesPullingTime="365.772124ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:55:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:55:09.203107758 +0000 UTC m=+1438.781200535" watchObservedRunningTime="2026-07-11 23:55:09.203256915 +0000 UTC m=+1438.781349690"
Jul 11 23:55:10.187510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:10.187413    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"
Jul 11 23:55:12.431188 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:12.431146    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp"]
Jul 11 23:55:12.433237 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:12.433220    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp"
Jul 11 23:55:12.435679 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:12.435655    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\""
Jul 11 23:55:12.441826 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:12.441792    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp"]
Jul 11 23:55:12.482898 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:12.482855    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12bce195-28d2-4bf1-934d-244a4f7b535d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp\" (UID: \"12bce195-28d2-4bf1-934d-244a4f7b535d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp"
Jul 11 23:55:12.584672 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:12.584635    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12bce195-28d2-4bf1-934d-244a4f7b535d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp\" (UID: \"12bce195-28d2-4bf1-934d-244a4f7b535d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp"
Jul 11 23:55:12.585370 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:12.585349    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12bce195-28d2-4bf1-934d-244a4f7b535d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp\" (UID: \"12bce195-28d2-4bf1-934d-244a4f7b535d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp"
Jul 11 23:55:12.744569 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:12.744491    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp"
Jul 11 23:55:12.923069 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:12.923043    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp"]
Jul 11 23:55:12.925883 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:55:12.925855    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12bce195_28d2_4bf1_934d_244a4f7b535d.slice/crio-5e1f735fffa8d1b37c1d503387b575dff65fcbbcffa0b5e948109d26dfb13c4c WatchSource:0}: Error finding container 5e1f735fffa8d1b37c1d503387b575dff65fcbbcffa0b5e948109d26dfb13c4c: Status 404 returned error can't find the container with id 5e1f735fffa8d1b37c1d503387b575dff65fcbbcffa0b5e948109d26dfb13c4c
Jul 11 23:55:13.201772 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:13.201740    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp" event={"ID":"12bce195-28d2-4bf1-934d-244a4f7b535d","Type":"ContainerStarted","Data":"c18c2a122b372dcfcc682a2c26fea2af8eeda7fcf285c283a713e9664e97b426"}
Jul 11 23:55:13.201953 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:13.201800    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp" event={"ID":"12bce195-28d2-4bf1-934d-244a4f7b535d","Type":"ContainerStarted","Data":"5e1f735fffa8d1b37c1d503387b575dff65fcbbcffa0b5e948109d26dfb13c4c"}
Jul 11 23:55:13.217028 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:13.216981    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp" podStartSLOduration=1.216964757 podStartE2EDuration="1.216964757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:55:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:55:13.214758612 +0000 UTC m=+1442.792851379" watchObservedRunningTime="2026-07-11 23:55:13.216964757 +0000 UTC m=+1442.795057533"
Jul 11 23:55:14.208016 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:14.207984    2569 generic.go:356] "Generic (PLEG): container finished" podID="12bce195-28d2-4bf1-934d-244a4f7b535d" containerID="c18c2a122b372dcfcc682a2c26fea2af8eeda7fcf285c283a713e9664e97b426" exitCode=7
Jul 11 23:55:14.208404 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:14.208062    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp" event={"ID":"12bce195-28d2-4bf1-934d-244a4f7b535d","Type":"ContainerDied","Data":"c18c2a122b372dcfcc682a2c26fea2af8eeda7fcf285c283a713e9664e97b426"}
Jul 11 23:55:14.208404 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:14.208211    2569 scope.go:117] "RemoveContainer" containerID="c18c2a122b372dcfcc682a2c26fea2af8eeda7fcf285c283a713e9664e97b426"
Jul 11 23:55:15.214070 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:15.214029    2569 generic.go:356] "Generic (PLEG): container finished" podID="12bce195-28d2-4bf1-934d-244a4f7b535d" containerID="5971aa911f9ec71032cc5d7e32cbd20a37c759ce74d9efce7c869ffa59ff5380" exitCode=7
Jul 11 23:55:15.214463 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:15.214085    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp" event={"ID":"12bce195-28d2-4bf1-934d-244a4f7b535d","Type":"ContainerDied","Data":"5971aa911f9ec71032cc5d7e32cbd20a37c759ce74d9efce7c869ffa59ff5380"}
Jul 11 23:55:15.214463 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:15.214144    2569 scope.go:117] "RemoveContainer" containerID="c18c2a122b372dcfcc682a2c26fea2af8eeda7fcf285c283a713e9664e97b426"
Jul 11 23:55:15.214463 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:15.214322    2569 scope.go:117] "RemoveContainer" containerID="5971aa911f9ec71032cc5d7e32cbd20a37c759ce74d9efce7c869ffa59ff5380"
Jul 11 23:55:15.214616 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:55:15.214536    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp_odh-ai-gateway-infra(12bce195-28d2-4bf1-934d-244a4f7b535d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp" podUID="12bce195-28d2-4bf1-934d-244a4f7b535d"
Jul 11 23:55:16.197214 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:16.197134    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"
Jul 11 23:55:16.221405 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:16.221374    2569 scope.go:117] "RemoveContainer" containerID="5971aa911f9ec71032cc5d7e32cbd20a37c759ce74d9efce7c869ffa59ff5380"
Jul 11 23:55:16.221771 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:55:16.221642    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp_odh-ai-gateway-infra(12bce195-28d2-4bf1-934d-244a4f7b535d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp" podUID="12bce195-28d2-4bf1-934d-244a4f7b535d"
Jul 11 23:55:19.237656 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:19.237624    2569 generic.go:356] "Generic (PLEG): container finished" podID="0e9550d2-3722-42d3-97dd-3477c52ae66f" containerID="8348920b86a5fd54f9e0cb6d2bb9934ee7693936f946688b58c70edd88600aa8" exitCode=0
Jul 11 23:55:19.238069 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:19.237700    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6" event={"ID":"0e9550d2-3722-42d3-97dd-3477c52ae66f","Type":"ContainerDied","Data":"8348920b86a5fd54f9e0cb6d2bb9934ee7693936f946688b58c70edd88600aa8"}
Jul 11 23:55:19.238140 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:19.238126    2569 scope.go:117] "RemoveContainer" containerID="8348920b86a5fd54f9e0cb6d2bb9934ee7693936f946688b58c70edd88600aa8"
Jul 11 23:55:19.239156 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:19.239122    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:55:19.239234 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:19.239213    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:55:19.239277 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:19.239263    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:55:20.244289 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:20.244256    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6" event={"ID":"0e9550d2-3722-42d3-97dd-3477c52ae66f","Type":"ContainerStarted","Data":"de5f8bdf4c66c054e5c167fb6a78713031c35c64d5df51fb861e50bc7c38b93b"}
Jul 11 23:55:20.575911 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:20.575863    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:20.577885 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:20.577845    2569 patch_prober.go:28] interesting pod/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Jul 11 23:55:20.578047 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:20.577909    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6" podUID="0e9550d2-3722-42d3-97dd-3477c52ae66f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Jul 11 23:55:21.576172 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:21.576124    2569 patch_prober.go:28] interesting pod/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Jul 11 23:55:21.576715 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:21.576193    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6" podUID="0e9550d2-3722-42d3-97dd-3477c52ae66f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Jul 11 23:55:22.575721 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:22.575682    2569 patch_prober.go:28] interesting pod/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Jul 11 23:55:22.576015 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:22.575980    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6" podUID="0e9550d2-3722-42d3-97dd-3477c52ae66f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Jul 11 23:55:23.176486 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:23.176443    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:23.575445 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:23.575381    2569 patch_prober.go:28] interesting pod/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body=
Jul 11 23:55:23.575660 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:23.575488    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6" podUID="0e9550d2-3722-42d3-97dd-3477c52ae66f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused"
Jul 11 23:55:24.579604 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:24.579572    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:24.580504 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:24.580486    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:55:26.978119 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:26.978087    2569 scope.go:117] "RemoveContainer" containerID="5971aa911f9ec71032cc5d7e32cbd20a37c759ce74d9efce7c869ffa59ff5380"
Jul 11 23:55:27.281098 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:27.281018    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp" event={"ID":"12bce195-28d2-4bf1-934d-244a4f7b535d","Type":"ContainerStarted","Data":"545b091c239d4cfd955dcfdb08456509c17bea8e70cd17be39d0703fd573a2c9"}
Jul 11 23:55:28.287018 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:28.286981    2569 generic.go:356] "Generic (PLEG): container finished" podID="12bce195-28d2-4bf1-934d-244a4f7b535d" containerID="545b091c239d4cfd955dcfdb08456509c17bea8e70cd17be39d0703fd573a2c9" exitCode=0
Jul 11 23:55:28.287446 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:28.287032    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp" event={"ID":"12bce195-28d2-4bf1-934d-244a4f7b535d","Type":"ContainerDied","Data":"545b091c239d4cfd955dcfdb08456509c17bea8e70cd17be39d0703fd573a2c9"}
Jul 11 23:55:28.287446 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:28.287118    2569 scope.go:117] "RemoveContainer" containerID="5971aa911f9ec71032cc5d7e32cbd20a37c759ce74d9efce7c869ffa59ff5380"
Jul 11 23:55:28.310640 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:28.310608    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp"]
Jul 11 23:55:29.430974 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:29.430950    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp"
Jul 11 23:55:29.539902 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:29.539868    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12bce195-28d2-4bf1-934d-244a4f7b535d-maas-api-service-ca\") pod \"12bce195-28d2-4bf1-934d-244a4f7b535d\" (UID: \"12bce195-28d2-4bf1-934d-244a4f7b535d\") "
Jul 11 23:55:29.540264 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:29.540234    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12bce195-28d2-4bf1-934d-244a4f7b535d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "12bce195-28d2-4bf1-934d-244a4f7b535d" (UID: "12bce195-28d2-4bf1-934d-244a4f7b535d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 23:55:29.641330 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:29.641244    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/12bce195-28d2-4bf1-934d-244a4f7b535d-maas-api-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:55:30.299235 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:30.299208    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp"
Jul 11 23:55:30.299405 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:30.299202    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp" event={"ID":"12bce195-28d2-4bf1-934d-244a4f7b535d","Type":"ContainerDied","Data":"5e1f735fffa8d1b37c1d503387b575dff65fcbbcffa0b5e948109d26dfb13c4c"}
Jul 11 23:55:30.299405 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:30.299338    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e1f735fffa8d1b37c1d503387b575dff65fcbbcffa0b5e948109d26dfb13c4c"
Jul 11 23:55:30.325282 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:30.325248    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp"]
Jul 11 23:55:30.330617 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:30.330591    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-2mnvp"]
Jul 11 23:55:30.982865 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:55:30.982835    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12bce195-28d2-4bf1-934d-244a4f7b535d" path="/var/lib/kubelet/pods/12bce195-28d2-4bf1-934d-244a4f7b535d/volumes"
Jul 11 23:56:02.469639 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:02.469598    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv"]
Jul 11 23:56:02.470102 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:02.469949    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12bce195-28d2-4bf1-934d-244a4f7b535d" containerName="revoke-keys"
Jul 11 23:56:02.470102 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:02.469960    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="12bce195-28d2-4bf1-934d-244a4f7b535d" containerName="revoke-keys"
Jul 11 23:56:02.470102 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:02.469970    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12bce195-28d2-4bf1-934d-244a4f7b535d" containerName="revoke-keys"
Jul 11 23:56:02.470102 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:02.469976    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="12bce195-28d2-4bf1-934d-244a4f7b535d" containerName="revoke-keys"
Jul 11 23:56:02.470102 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:02.470044    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="12bce195-28d2-4bf1-934d-244a4f7b535d" containerName="revoke-keys"
Jul 11 23:56:02.470102 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:02.470052    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="12bce195-28d2-4bf1-934d-244a4f7b535d" containerName="revoke-keys"
Jul 11 23:56:02.472080 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:02.472062    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv"
Jul 11 23:56:02.474582 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:02.474560    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\""
Jul 11 23:56:02.479639 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:02.479610    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv"]
Jul 11 23:56:02.536315 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:02.536276    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43341577-1d7f-4a0e-b0dd-a6ef1f73cfef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv\" (UID: \"43341577-1d7f-4a0e-b0dd-a6ef1f73cfef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv"
Jul 11 23:56:02.637493 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:02.637414    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43341577-1d7f-4a0e-b0dd-a6ef1f73cfef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv\" (UID: \"43341577-1d7f-4a0e-b0dd-a6ef1f73cfef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv"
Jul 11 23:56:02.638078 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:02.638058    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43341577-1d7f-4a0e-b0dd-a6ef1f73cfef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv\" (UID: \"43341577-1d7f-4a0e-b0dd-a6ef1f73cfef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv"
Jul 11 23:56:02.784668 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:02.784579    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv"
Jul 11 23:56:02.917331 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:02.917309    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv"]
Jul 11 23:56:02.920145 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:56:02.920111    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43341577_1d7f_4a0e_b0dd_a6ef1f73cfef.slice/crio-8d8830e60601d324ad2ba130253ff9975840b5bc4154a202bf15a93afd328f91 WatchSource:0}: Error finding container 8d8830e60601d324ad2ba130253ff9975840b5bc4154a202bf15a93afd328f91: Status 404 returned error can't find the container with id 8d8830e60601d324ad2ba130253ff9975840b5bc4154a202bf15a93afd328f91
Jul 11 23:56:03.449474 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:03.449355    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv" event={"ID":"43341577-1d7f-4a0e-b0dd-a6ef1f73cfef","Type":"ContainerStarted","Data":"b96faa17164bf9560d499dcc8ffdb45fd2c7d940f05cca994766682e94b47b6c"}
Jul 11 23:56:03.449474 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:03.449406    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv" event={"ID":"43341577-1d7f-4a0e-b0dd-a6ef1f73cfef","Type":"ContainerStarted","Data":"8d8830e60601d324ad2ba130253ff9975840b5bc4154a202bf15a93afd328f91"}
Jul 11 23:56:03.464949 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:03.464887    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv" podStartSLOduration=1.464869526 podStartE2EDuration="1.464869526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:56:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:56:03.463130793 +0000 UTC m=+1493.041223569" watchObservedRunningTime="2026-07-11 23:56:03.464869526 +0000 UTC m=+1493.042962302"
Jul 11 23:56:04.455888 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:04.455851    2569 generic.go:356] "Generic (PLEG): container finished" podID="43341577-1d7f-4a0e-b0dd-a6ef1f73cfef" containerID="b96faa17164bf9560d499dcc8ffdb45fd2c7d940f05cca994766682e94b47b6c" exitCode=0
Jul 11 23:56:04.456333 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:04.455928    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv" event={"ID":"43341577-1d7f-4a0e-b0dd-a6ef1f73cfef","Type":"ContainerDied","Data":"b96faa17164bf9560d499dcc8ffdb45fd2c7d940f05cca994766682e94b47b6c"}
Jul 11 23:56:05.597150 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:05.597124    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv"
Jul 11 23:56:05.666420 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:05.666386    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43341577-1d7f-4a0e-b0dd-a6ef1f73cfef-maas-api-service-ca\") pod \"43341577-1d7f-4a0e-b0dd-a6ef1f73cfef\" (UID: \"43341577-1d7f-4a0e-b0dd-a6ef1f73cfef\") "
Jul 11 23:56:05.666739 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:05.666719    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43341577-1d7f-4a0e-b0dd-a6ef1f73cfef-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "43341577-1d7f-4a0e-b0dd-a6ef1f73cfef" (UID: "43341577-1d7f-4a0e-b0dd-a6ef1f73cfef"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 23:56:05.767838 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:05.767753    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43341577-1d7f-4a0e-b0dd-a6ef1f73cfef-maas-api-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:56:06.471702 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:06.471673    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv"
Jul 11 23:56:06.471896 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:06.471666    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv" event={"ID":"43341577-1d7f-4a0e-b0dd-a6ef1f73cfef","Type":"ContainerDied","Data":"8d8830e60601d324ad2ba130253ff9975840b5bc4154a202bf15a93afd328f91"}
Jul 11 23:56:06.471896 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:06.471815    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d8830e60601d324ad2ba130253ff9975840b5bc4154a202bf15a93afd328f91"
Jul 11 23:56:11.863146 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:11.863116    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 11 23:56:11.865765 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:11.865741    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 11 23:56:12.793530 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:12.793490    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"]
Jul 11 23:56:12.793753 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:12.793721    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5" podUID="ff0e692a-67bc-474f-ae49-09cc05d2be67" containerName="maas-api" containerID="cri-o://66303779c42ac124be50984019438eb144af1da82644c04a27b4278275805506" gracePeriod=30
Jul 11 23:56:13.047754 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.047698    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"
Jul 11 23:56:13.128867 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.128828    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgn6v\" (UniqueName: \"kubernetes.io/projected/ff0e692a-67bc-474f-ae49-09cc05d2be67-kube-api-access-xgn6v\") pod \"ff0e692a-67bc-474f-ae49-09cc05d2be67\" (UID: \"ff0e692a-67bc-474f-ae49-09cc05d2be67\") "
Jul 11 23:56:13.129052 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.128881    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff0e692a-67bc-474f-ae49-09cc05d2be67-openshift-service-ca\") pod \"ff0e692a-67bc-474f-ae49-09cc05d2be67\" (UID: \"ff0e692a-67bc-474f-ae49-09cc05d2be67\") "
Jul 11 23:56:13.129052 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.128944    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff0e692a-67bc-474f-ae49-09cc05d2be67-maas-api-tls\") pod \"ff0e692a-67bc-474f-ae49-09cc05d2be67\" (UID: \"ff0e692a-67bc-474f-ae49-09cc05d2be67\") "
Jul 11 23:56:13.129297 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.129271    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff0e692a-67bc-474f-ae49-09cc05d2be67-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ff0e692a-67bc-474f-ae49-09cc05d2be67" (UID: "ff0e692a-67bc-474f-ae49-09cc05d2be67"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 23:56:13.131168 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.131144    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff0e692a-67bc-474f-ae49-09cc05d2be67-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ff0e692a-67bc-474f-ae49-09cc05d2be67" (UID: "ff0e692a-67bc-474f-ae49-09cc05d2be67"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 23:56:13.131274 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.131214    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff0e692a-67bc-474f-ae49-09cc05d2be67-kube-api-access-xgn6v" (OuterVolumeSpecName: "kube-api-access-xgn6v") pod "ff0e692a-67bc-474f-ae49-09cc05d2be67" (UID: "ff0e692a-67bc-474f-ae49-09cc05d2be67"). InnerVolumeSpecName "kube-api-access-xgn6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 23:56:13.229641 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.229603    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgn6v\" (UniqueName: \"kubernetes.io/projected/ff0e692a-67bc-474f-ae49-09cc05d2be67-kube-api-access-xgn6v\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:56:13.229641 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.229634    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ff0e692a-67bc-474f-ae49-09cc05d2be67-openshift-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:56:13.229641 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.229645    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ff0e692a-67bc-474f-ae49-09cc05d2be67-maas-api-tls\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:56:13.509113 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.509026    2569 generic.go:356] "Generic (PLEG): container finished" podID="ff0e692a-67bc-474f-ae49-09cc05d2be67" containerID="66303779c42ac124be50984019438eb144af1da82644c04a27b4278275805506" exitCode=0
Jul 11 23:56:13.509113 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.509066    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"
Jul 11 23:56:13.509113 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.509091    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5" event={"ID":"ff0e692a-67bc-474f-ae49-09cc05d2be67","Type":"ContainerDied","Data":"66303779c42ac124be50984019438eb144af1da82644c04a27b4278275805506"}
Jul 11 23:56:13.509349 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.509132    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5" event={"ID":"ff0e692a-67bc-474f-ae49-09cc05d2be67","Type":"ContainerDied","Data":"a482caa78e69a21aee828e261a5a6df5b4cc496ad5fc0e0e421929686bce081a"}
Jul 11 23:56:13.509349 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.509153    2569 scope.go:117] "RemoveContainer" containerID="66303779c42ac124be50984019438eb144af1da82644c04a27b4278275805506"
Jul 11 23:56:13.518606 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.518589    2569 scope.go:117] "RemoveContainer" containerID="66303779c42ac124be50984019438eb144af1da82644c04a27b4278275805506"
Jul 11 23:56:13.518849 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:56:13.518832    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66303779c42ac124be50984019438eb144af1da82644c04a27b4278275805506\": container with ID starting with 66303779c42ac124be50984019438eb144af1da82644c04a27b4278275805506 not found: ID does not exist" containerID="66303779c42ac124be50984019438eb144af1da82644c04a27b4278275805506"
Jul 11 23:56:13.518890 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.518858    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66303779c42ac124be50984019438eb144af1da82644c04a27b4278275805506"} err="failed to get container status \"66303779c42ac124be50984019438eb144af1da82644c04a27b4278275805506\": rpc error: code = NotFound desc = could not find container \"66303779c42ac124be50984019438eb144af1da82644c04a27b4278275805506\": container with ID starting with 66303779c42ac124be50984019438eb144af1da82644c04a27b4278275805506 not found: ID does not exist"
Jul 11 23:56:13.530316 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.530294    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"]
Jul 11 23:56:13.533662 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:13.533639    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb1c09fd-77856f775d-pvwq5"]
Jul 11 23:56:14.983980 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:14.983942    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff0e692a-67bc-474f-ae49-09cc05d2be67" path="/var/lib/kubelet/pods/ff0e692a-67bc-474f-ae49-09cc05d2be67/volumes"
Jul 11 23:56:27.710108 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:27.710067    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"]
Jul 11 23:56:27.710612 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:27.710449    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6" podUID="0e9550d2-3722-42d3-97dd-3477c52ae66f" containerName="istio-proxy" containerID="cri-o://de5f8bdf4c66c054e5c167fb6a78713031c35c64d5df51fb861e50bc7c38b93b" gracePeriod=30
Jul 11 23:56:28.262933 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.262894    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"]
Jul 11 23:56:28.263483 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.263462    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43341577-1d7f-4a0e-b0dd-a6ef1f73cfef" containerName="revoke-keys"
Jul 11 23:56:28.263483 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.263485    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="43341577-1d7f-4a0e-b0dd-a6ef1f73cfef" containerName="revoke-keys"
Jul 11 23:56:28.263658 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.263514    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12bce195-28d2-4bf1-934d-244a4f7b535d" containerName="revoke-keys"
Jul 11 23:56:28.263658 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.263524    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="12bce195-28d2-4bf1-934d-244a4f7b535d" containerName="revoke-keys"
Jul 11 23:56:28.263658 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.263536    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff0e692a-67bc-474f-ae49-09cc05d2be67" containerName="maas-api"
Jul 11 23:56:28.263658 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.263545    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff0e692a-67bc-474f-ae49-09cc05d2be67" containerName="maas-api"
Jul 11 23:56:28.263658 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.263636    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="43341577-1d7f-4a0e-b0dd-a6ef1f73cfef" containerName="revoke-keys"
Jul 11 23:56:28.263658 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.263648    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff0e692a-67bc-474f-ae49-09cc05d2be67" containerName="maas-api"
Jul 11 23:56:28.263658 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.263658    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="12bce195-28d2-4bf1-934d-244a4f7b535d" containerName="revoke-keys"
Jul 11 23:56:28.268767 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.268742    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.271172 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.271150    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-da8e2e5f-gw-openshift-default-dockercfg-gz4cf\""
Jul 11 23:56:28.275386 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.275360    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"]
Jul 11 23:56:28.366767 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.366721    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-credential-socket\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.366767 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.366771    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-workload-certs\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.367006 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.366806    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f87928b-cb4e-467e-8115-161e66a18c7a-istiod-ca-cert\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.367006 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.366871    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-envoy\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.367006 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.366911    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-workload-socket\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.367123 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.367039    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqf4s\" (UniqueName: \"kubernetes.io/projected/2f87928b-cb4e-467e-8115-161e66a18c7a-kube-api-access-bqf4s\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.367168 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.367138    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-podinfo\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.367208 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.367177    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-token\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.367300 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.367264    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-data\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.468865 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.468810    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqf4s\" (UniqueName: \"kubernetes.io/projected/2f87928b-cb4e-467e-8115-161e66a18c7a-kube-api-access-bqf4s\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.469036 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.468917    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-podinfo\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.469036 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.468956    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-token\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.469036 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.468996    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-data\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.469036 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.469028    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-credential-socket\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.469240 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.469051    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-workload-certs\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.469240 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.469082    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f87928b-cb4e-467e-8115-161e66a18c7a-istiod-ca-cert\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.469240 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.469109    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-envoy\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.469240 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.469136    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-workload-socket\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.469514 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.469488    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-data\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.469595 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.469523    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-credential-socket\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.469637 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.469611    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-workload-certs\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.469714 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.469694    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-workload-socket\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.469891 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.469867    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f87928b-cb4e-467e-8115-161e66a18c7a-istiod-ca-cert\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.471491 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.471472    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-envoy\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.471688 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.471671    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-podinfo\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.476621 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.476598    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-token\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.476690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.476635    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqf4s\" (UniqueName: \"kubernetes.io/projected/2f87928b-cb4e-467e-8115-161e66a18c7a-kube-api-access-bqf4s\") pod \"e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") " pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.580614 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.580585    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:28.719050 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.719020    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"]
Jul 11 23:56:28.720978 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:56:28.720951    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f87928b_cb4e_467e_8115_161e66a18c7a.slice/crio-6db003a1baa7f08ace0de229a66fd7eeccaecba759624b6abc7d46af958c7447 WatchSource:0}: Error finding container 6db003a1baa7f08ace0de229a66fd7eeccaecba759624b6abc7d46af958c7447: Status 404 returned error can't find the container with id 6db003a1baa7f08ace0de229a66fd7eeccaecba759624b6abc7d46af958c7447
Jul 11 23:56:28.723209 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.723177    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:56:28.723291 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.723245    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:56:28.723291 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:28.723275    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:56:29.585358 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:29.585315    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk" event={"ID":"2f87928b-cb4e-467e-8115-161e66a18c7a","Type":"ContainerStarted","Data":"0fb44a6e4d6bdcdccadf7d3439c6dfb178501f0b921f6925247debc45d7de0c6"}
Jul 11 23:56:29.585550 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:29.585411    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk" event={"ID":"2f87928b-cb4e-467e-8115-161e66a18c7a","Type":"ContainerStarted","Data":"6db003a1baa7f08ace0de229a66fd7eeccaecba759624b6abc7d46af958c7447"}
Jul 11 23:56:29.604809 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:29.604751    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk" podStartSLOduration=1.6047347090000001 podStartE2EDuration="1.604734709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:56:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:56:29.603484211 +0000 UTC m=+1519.181576988" watchObservedRunningTime="2026-07-11 23:56:29.604734709 +0000 UTC m=+1519.182827484"
Jul 11 23:56:29.711286 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:29.711249    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"]
Jul 11 23:56:29.714966 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:29.714944    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"
Jul 11 23:56:29.717629 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:29.717602    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-da8e2e5f-serving-cert\""
Jul 11 23:56:29.724779 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:29.724750    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"]
Jul 11 23:56:29.781636 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:29.781605    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cc9f480-873b-425b-9ec6-8c25008b5a59-maas-api-tls\") pod \"maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n\" (UID: \"8cc9f480-873b-425b-9ec6-8c25008b5a59\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"
Jul 11 23:56:29.781797 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:29.781650    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cc9f480-873b-425b-9ec6-8c25008b5a59-openshift-service-ca\") pod \"maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n\" (UID: \"8cc9f480-873b-425b-9ec6-8c25008b5a59\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"
Jul 11 23:56:29.781797 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:29.781714    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgck7\" (UniqueName: \"kubernetes.io/projected/8cc9f480-873b-425b-9ec6-8c25008b5a59-kube-api-access-tgck7\") pod \"maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n\" (UID: \"8cc9f480-873b-425b-9ec6-8c25008b5a59\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"
Jul 11 23:56:29.883023 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:29.882239    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cc9f480-873b-425b-9ec6-8c25008b5a59-maas-api-tls\") pod \"maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n\" (UID: \"8cc9f480-873b-425b-9ec6-8c25008b5a59\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"
Jul 11 23:56:29.883023 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:29.882321    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cc9f480-873b-425b-9ec6-8c25008b5a59-openshift-service-ca\") pod \"maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n\" (UID: \"8cc9f480-873b-425b-9ec6-8c25008b5a59\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"
Jul 11 23:56:29.883023 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:29.882391    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgck7\" (UniqueName: \"kubernetes.io/projected/8cc9f480-873b-425b-9ec6-8c25008b5a59-kube-api-access-tgck7\") pod \"maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n\" (UID: \"8cc9f480-873b-425b-9ec6-8c25008b5a59\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"
Jul 11 23:56:29.883523 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:29.883496    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cc9f480-873b-425b-9ec6-8c25008b5a59-openshift-service-ca\") pod \"maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n\" (UID: \"8cc9f480-873b-425b-9ec6-8c25008b5a59\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"
Jul 11 23:56:29.885987 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:29.885958    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cc9f480-873b-425b-9ec6-8c25008b5a59-maas-api-tls\") pod \"maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n\" (UID: \"8cc9f480-873b-425b-9ec6-8c25008b5a59\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"
Jul 11 23:56:29.890199 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:29.890171    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgck7\" (UniqueName: \"kubernetes.io/projected/8cc9f480-873b-425b-9ec6-8c25008b5a59-kube-api-access-tgck7\") pod \"maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n\" (UID: \"8cc9f480-873b-425b-9ec6-8c25008b5a59\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"
Jul 11 23:56:30.028593 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:30.028560    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"
Jul 11 23:56:30.166967 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:30.166941    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"]
Jul 11 23:56:30.168712 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:56:30.168684    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cc9f480_873b_425b_9ec6_8c25008b5a59.slice/crio-917a56dad8d111d367cf8ddc874faead3c065e19cb694c22e6e956d6768aba7e WatchSource:0}: Error finding container 917a56dad8d111d367cf8ddc874faead3c065e19cb694c22e6e956d6768aba7e: Status 404 returned error can't find the container with id 917a56dad8d111d367cf8ddc874faead3c065e19cb694c22e6e956d6768aba7e
Jul 11 23:56:30.581498 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:30.581473    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:30.586596 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:30.586566    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:30.592220 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:30.592193    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n" event={"ID":"8cc9f480-873b-425b-9ec6-8c25008b5a59","Type":"ContainerStarted","Data":"65423cb194725a2134c14ce0ce2d118c67e9b4ffc7f7c122341700b6ae304a8a"}
Jul 11 23:56:30.592321 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:30.592239    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n" event={"ID":"8cc9f480-873b-425b-9ec6-8c25008b5a59","Type":"ContainerStarted","Data":"917a56dad8d111d367cf8ddc874faead3c065e19cb694c22e6e956d6768aba7e"}
Jul 11 23:56:30.592408 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:30.592395    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"
Jul 11 23:56:30.592484 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:30.592445    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:30.593093 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:30.593075    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:30.636205 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:30.636149    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n" podStartSLOduration=1.298203525 podStartE2EDuration="1.636132401s" totalImagesPullingTime="337.928876ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:56:30.635683241 +0000 UTC m=+1520.213776017" watchObservedRunningTime="2026-07-11 23:56:30.636132401 +0000 UTC m=+1520.214225175"
Jul 11 23:56:32.969285 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:32.969261    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:56:33.116027 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.115991    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-workload-certs\") pod \"0e9550d2-3722-42d3-97dd-3477c52ae66f\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") "
Jul 11 23:56:33.116214 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.116048    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-podinfo\") pod \"0e9550d2-3722-42d3-97dd-3477c52ae66f\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") "
Jul 11 23:56:33.116214 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.116084    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-envoy\") pod \"0e9550d2-3722-42d3-97dd-3477c52ae66f\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") "
Jul 11 23:56:33.116214 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.116106    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-token\") pod \"0e9550d2-3722-42d3-97dd-3477c52ae66f\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") "
Jul 11 23:56:33.116214 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.116135    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e9550d2-3722-42d3-97dd-3477c52ae66f-istiod-ca-cert\") pod \"0e9550d2-3722-42d3-97dd-3477c52ae66f\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") "
Jul 11 23:56:33.116214 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.116169    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7d2z\" (UniqueName: \"kubernetes.io/projected/0e9550d2-3722-42d3-97dd-3477c52ae66f-kube-api-access-w7d2z\") pod \"0e9550d2-3722-42d3-97dd-3477c52ae66f\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") "
Jul 11 23:56:33.116214 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.116193    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-workload-socket\") pod \"0e9550d2-3722-42d3-97dd-3477c52ae66f\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") "
Jul 11 23:56:33.116564 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.116226    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-data\") pod \"0e9550d2-3722-42d3-97dd-3477c52ae66f\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") "
Jul 11 23:56:33.116564 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.116307    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-credential-socket\") pod \"0e9550d2-3722-42d3-97dd-3477c52ae66f\" (UID: \"0e9550d2-3722-42d3-97dd-3477c52ae66f\") "
Jul 11 23:56:33.116564 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.116318    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0e9550d2-3722-42d3-97dd-3477c52ae66f" (UID: "0e9550d2-3722-42d3-97dd-3477c52ae66f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 23:56:33.116724 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.116597    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e9550d2-3722-42d3-97dd-3477c52ae66f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0e9550d2-3722-42d3-97dd-3477c52ae66f" (UID: "0e9550d2-3722-42d3-97dd-3477c52ae66f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 23:56:33.116724 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.116606    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0e9550d2-3722-42d3-97dd-3477c52ae66f" (UID: "0e9550d2-3722-42d3-97dd-3477c52ae66f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 23:56:33.116724 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.116709    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-workload-certs\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:56:33.116883 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.116728    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e9550d2-3722-42d3-97dd-3477c52ae66f-istiod-ca-cert\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:56:33.116883 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.116741    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-workload-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:56:33.116883 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.116829    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0e9550d2-3722-42d3-97dd-3477c52ae66f" (UID: "0e9550d2-3722-42d3-97dd-3477c52ae66f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 23:56:33.117159 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.117139    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-data" (OuterVolumeSpecName: "istio-data") pod "0e9550d2-3722-42d3-97dd-3477c52ae66f" (UID: "0e9550d2-3722-42d3-97dd-3477c52ae66f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 23:56:33.118623 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.118594    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0e9550d2-3722-42d3-97dd-3477c52ae66f" (UID: "0e9550d2-3722-42d3-97dd-3477c52ae66f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 11 23:56:33.118713 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.118604    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-token" (OuterVolumeSpecName: "istio-token") pod "0e9550d2-3722-42d3-97dd-3477c52ae66f" (UID: "0e9550d2-3722-42d3-97dd-3477c52ae66f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 23:56:33.118769 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.118723    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e9550d2-3722-42d3-97dd-3477c52ae66f-kube-api-access-w7d2z" (OuterVolumeSpecName: "kube-api-access-w7d2z") pod "0e9550d2-3722-42d3-97dd-3477c52ae66f" (UID: "0e9550d2-3722-42d3-97dd-3477c52ae66f"). InnerVolumeSpecName "kube-api-access-w7d2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 23:56:33.118769 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.118736    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0e9550d2-3722-42d3-97dd-3477c52ae66f" (UID: "0e9550d2-3722-42d3-97dd-3477c52ae66f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 23:56:33.218166 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.218130    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-envoy\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:56:33.218166 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.218162    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-token\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:56:33.218447 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.218176    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7d2z\" (UniqueName: \"kubernetes.io/projected/0e9550d2-3722-42d3-97dd-3477c52ae66f-kube-api-access-w7d2z\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:56:33.218447 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.218186    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-data\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:56:33.218447 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.218195    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e9550d2-3722-42d3-97dd-3477c52ae66f-credential-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:56:33.218447 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.218202    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e9550d2-3722-42d3-97dd-3477c52ae66f-istio-podinfo\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:56:33.609314 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.609276    2569 generic.go:356] "Generic (PLEG): container finished" podID="0e9550d2-3722-42d3-97dd-3477c52ae66f" containerID="de5f8bdf4c66c054e5c167fb6a78713031c35c64d5df51fb861e50bc7c38b93b" exitCode=0
Jul 11 23:56:33.609525 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.609326    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"
Jul 11 23:56:33.609525 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.609351    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6" event={"ID":"0e9550d2-3722-42d3-97dd-3477c52ae66f","Type":"ContainerDied","Data":"de5f8bdf4c66c054e5c167fb6a78713031c35c64d5df51fb861e50bc7c38b93b"}
Jul 11 23:56:33.609525 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.609401    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6" event={"ID":"0e9550d2-3722-42d3-97dd-3477c52ae66f","Type":"ContainerDied","Data":"9a1b80aa0f3368c06c07bb09a1493892850ac5b3dd514452d631ee8bad78a324"}
Jul 11 23:56:33.609525 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.609437    2569 scope.go:117] "RemoveContainer" containerID="de5f8bdf4c66c054e5c167fb6a78713031c35c64d5df51fb861e50bc7c38b93b"
Jul 11 23:56:33.618634 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.618617    2569 scope.go:117] "RemoveContainer" containerID="8348920b86a5fd54f9e0cb6d2bb9934ee7693936f946688b58c70edd88600aa8"
Jul 11 23:56:33.626227 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.626204    2569 scope.go:117] "RemoveContainer" containerID="de5f8bdf4c66c054e5c167fb6a78713031c35c64d5df51fb861e50bc7c38b93b"
Jul 11 23:56:33.626463 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:56:33.626446    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de5f8bdf4c66c054e5c167fb6a78713031c35c64d5df51fb861e50bc7c38b93b\": container with ID starting with de5f8bdf4c66c054e5c167fb6a78713031c35c64d5df51fb861e50bc7c38b93b not found: ID does not exist" containerID="de5f8bdf4c66c054e5c167fb6a78713031c35c64d5df51fb861e50bc7c38b93b"
Jul 11 23:56:33.626518 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.626472    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de5f8bdf4c66c054e5c167fb6a78713031c35c64d5df51fb861e50bc7c38b93b"} err="failed to get container status \"de5f8bdf4c66c054e5c167fb6a78713031c35c64d5df51fb861e50bc7c38b93b\": rpc error: code = NotFound desc = could not find container \"de5f8bdf4c66c054e5c167fb6a78713031c35c64d5df51fb861e50bc7c38b93b\": container with ID starting with de5f8bdf4c66c054e5c167fb6a78713031c35c64d5df51fb861e50bc7c38b93b not found: ID does not exist"
Jul 11 23:56:33.626518 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.626487    2569 scope.go:117] "RemoveContainer" containerID="8348920b86a5fd54f9e0cb6d2bb9934ee7693936f946688b58c70edd88600aa8"
Jul 11 23:56:33.626727 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:56:33.626708    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8348920b86a5fd54f9e0cb6d2bb9934ee7693936f946688b58c70edd88600aa8\": container with ID starting with 8348920b86a5fd54f9e0cb6d2bb9934ee7693936f946688b58c70edd88600aa8 not found: ID does not exist" containerID="8348920b86a5fd54f9e0cb6d2bb9934ee7693936f946688b58c70edd88600aa8"
Jul 11 23:56:33.626781 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.626730    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8348920b86a5fd54f9e0cb6d2bb9934ee7693936f946688b58c70edd88600aa8"} err="failed to get container status \"8348920b86a5fd54f9e0cb6d2bb9934ee7693936f946688b58c70edd88600aa8\": rpc error: code = NotFound desc = could not find container \"8348920b86a5fd54f9e0cb6d2bb9934ee7693936f946688b58c70edd88600aa8\": container with ID starting with 8348920b86a5fd54f9e0cb6d2bb9934ee7693936f946688b58c70edd88600aa8 not found: ID does not exist"
Jul 11 23:56:33.631492 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.631469    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"]
Jul 11 23:56:33.638610 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:33.638587    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-eb1c09fd-openshift-default-55d4888b59-jl4k6"]
Jul 11 23:56:34.007038 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:34.006963    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v"]
Jul 11 23:56:34.007371 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:34.007334    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e9550d2-3722-42d3-97dd-3477c52ae66f" containerName="istio-proxy"
Jul 11 23:56:34.007371 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:34.007347    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e9550d2-3722-42d3-97dd-3477c52ae66f" containerName="istio-proxy"
Jul 11 23:56:34.007464 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:34.007455    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e9550d2-3722-42d3-97dd-3477c52ae66f" containerName="istio-proxy"
Jul 11 23:56:34.007497 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:34.007465    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e9550d2-3722-42d3-97dd-3477c52ae66f" containerName="istio-proxy"
Jul 11 23:56:34.010546 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:34.010527    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v"
Jul 11 23:56:34.012966 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:34.012942    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\""
Jul 11 23:56:34.016384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:34.016362    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v"]
Jul 11 23:56:34.124338 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:34.124298    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d515225-6f2e-4405-99e5-a6e755f831c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v\" (UID: \"0d515225-6f2e-4405-99e5-a6e755f831c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v"
Jul 11 23:56:34.225744 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:34.225707    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d515225-6f2e-4405-99e5-a6e755f831c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v\" (UID: \"0d515225-6f2e-4405-99e5-a6e755f831c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v"
Jul 11 23:56:34.226370 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:34.226347    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d515225-6f2e-4405-99e5-a6e755f831c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v\" (UID: \"0d515225-6f2e-4405-99e5-a6e755f831c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v"
Jul 11 23:56:34.323937 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:34.323907    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v"
Jul 11 23:56:34.456441 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:34.456398    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v"]
Jul 11 23:56:34.459208 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:56:34.459182    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d515225_6f2e_4405_99e5_a6e755f831c8.slice/crio-2437d6e446d30a4d7a449e9664bfe9c17ea576b795100a7796f762673b44a82e WatchSource:0}: Error finding container 2437d6e446d30a4d7a449e9664bfe9c17ea576b795100a7796f762673b44a82e: Status 404 returned error can't find the container with id 2437d6e446d30a4d7a449e9664bfe9c17ea576b795100a7796f762673b44a82e
Jul 11 23:56:34.616233 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:34.616148    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v" event={"ID":"0d515225-6f2e-4405-99e5-a6e755f831c8","Type":"ContainerStarted","Data":"2437d6e446d30a4d7a449e9664bfe9c17ea576b795100a7796f762673b44a82e"}
Jul 11 23:56:34.983460 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:34.983371    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e9550d2-3722-42d3-97dd-3477c52ae66f" path="/var/lib/kubelet/pods/0e9550d2-3722-42d3-97dd-3477c52ae66f/volumes"
Jul 11 23:56:35.622837 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:35.622802    2569 generic.go:356] "Generic (PLEG): container finished" podID="0d515225-6f2e-4405-99e5-a6e755f831c8" containerID="c34ffc7b76a31c4e69bfea9a2c96fa1bd61c5c6b12072c35d0d1c0d65ba70192" exitCode=7
Jul 11 23:56:35.623243 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:35.622890    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v" event={"ID":"0d515225-6f2e-4405-99e5-a6e755f831c8","Type":"ContainerDied","Data":"c34ffc7b76a31c4e69bfea9a2c96fa1bd61c5c6b12072c35d0d1c0d65ba70192"}
Jul 11 23:56:35.623243 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:35.623069    2569 scope.go:117] "RemoveContainer" containerID="c34ffc7b76a31c4e69bfea9a2c96fa1bd61c5c6b12072c35d0d1c0d65ba70192"
Jul 11 23:56:36.602216 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:36.602190    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"
Jul 11 23:56:36.630131 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:36.630098    2569 generic.go:356] "Generic (PLEG): container finished" podID="0d515225-6f2e-4405-99e5-a6e755f831c8" containerID="650e539a5644f293067528f3b430e384aad80ca331769666b92bc57e6edbb987" exitCode=7
Jul 11 23:56:36.630573 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:36.630160    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v" event={"ID":"0d515225-6f2e-4405-99e5-a6e755f831c8","Type":"ContainerDied","Data":"650e539a5644f293067528f3b430e384aad80ca331769666b92bc57e6edbb987"}
Jul 11 23:56:36.630573 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:36.630208    2569 scope.go:117] "RemoveContainer" containerID="c34ffc7b76a31c4e69bfea9a2c96fa1bd61c5c6b12072c35d0d1c0d65ba70192"
Jul 11 23:56:36.630573 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:36.630493    2569 scope.go:117] "RemoveContainer" containerID="650e539a5644f293067528f3b430e384aad80ca331769666b92bc57e6edbb987"
Jul 11 23:56:36.630756 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:56:36.630721    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v_odh-ai-gateway-infra(0d515225-6f2e-4405-99e5-a6e755f831c8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v" podUID="0d515225-6f2e-4405-99e5-a6e755f831c8"
Jul 11 23:56:37.636267 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:37.636237    2569 scope.go:117] "RemoveContainer" containerID="650e539a5644f293067528f3b430e384aad80ca331769666b92bc57e6edbb987"
Jul 11 23:56:37.636637 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:56:37.636532    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v_odh-ai-gateway-infra(0d515225-6f2e-4405-99e5-a6e755f831c8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v" podUID="0d515225-6f2e-4405-99e5-a6e755f831c8"
Jul 11 23:56:42.663461 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:42.663368    2569 generic.go:356] "Generic (PLEG): container finished" podID="2f87928b-cb4e-467e-8115-161e66a18c7a" containerID="0fb44a6e4d6bdcdccadf7d3439c6dfb178501f0b921f6925247debc45d7de0c6" exitCode=0
Jul 11 23:56:42.663910 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:42.663458    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk" event={"ID":"2f87928b-cb4e-467e-8115-161e66a18c7a","Type":"ContainerDied","Data":"0fb44a6e4d6bdcdccadf7d3439c6dfb178501f0b921f6925247debc45d7de0c6"}
Jul 11 23:56:42.663910 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:42.663899    2569 scope.go:117] "RemoveContainer" containerID="0fb44a6e4d6bdcdccadf7d3439c6dfb178501f0b921f6925247debc45d7de0c6"
Jul 11 23:56:42.664912 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:42.664879    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:56:42.665028 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:42.664953    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:56:42.665028 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:42.664986    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:56:43.674655 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:43.674612    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk" event={"ID":"2f87928b-cb4e-467e-8115-161e66a18c7a","Type":"ContainerStarted","Data":"cc4d562c2bd469a1d675f238bbb5402da915c1681fd72b6ba703a98393acd091"}
Jul 11 23:56:44.582519 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:44.582485    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:44.582855 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:44.582833    2569 patch_prober.go:28] interesting pod/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 11 23:56:44.583011 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:44.582986    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk" podUID="2f87928b-cb4e-467e-8115-161e66a18c7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 11 23:56:45.581297 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:45.581248    2569 patch_prober.go:28] interesting pod/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 11 23:56:45.581825 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:45.581323    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk" podUID="2f87928b-cb4e-467e-8115-161e66a18c7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 11 23:56:45.593082 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:45.593051    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:46.581945 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:46.581905    2569 patch_prober.go:28] interesting pod/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 11 23:56:46.582573 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:46.582534    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk" podUID="2f87928b-cb4e-467e-8115-161e66a18c7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 11 23:56:47.581586 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:47.581544    2569 patch_prober.go:28] interesting pod/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused" start-of-body=
Jul 11 23:56:47.581758 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:47.581634    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk" podUID="2f87928b-cb4e-467e-8115-161e66a18c7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.38:15021/healthz/ready\": dial tcp 10.133.0.38:15021: connect: connection refused"
Jul 11 23:56:48.585340 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:48.585303    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:48.586335 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:48.586314    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:56:51.977044 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:51.977015    2569 scope.go:117] "RemoveContainer" containerID="650e539a5644f293067528f3b430e384aad80ca331769666b92bc57e6edbb987"
Jul 11 23:56:52.722691 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:52.722655    2569 generic.go:356] "Generic (PLEG): container finished" podID="0d515225-6f2e-4405-99e5-a6e755f831c8" containerID="097cb96ead74cf4b44765fcb4f665eb9d56f5b5866f783c717acdb439fe598c1" exitCode=0
Jul 11 23:56:52.722891 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:52.722729    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v" event={"ID":"0d515225-6f2e-4405-99e5-a6e755f831c8","Type":"ContainerDied","Data":"097cb96ead74cf4b44765fcb4f665eb9d56f5b5866f783c717acdb439fe598c1"}
Jul 11 23:56:52.722891 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:52.722819    2569 scope.go:117] "RemoveContainer" containerID="650e539a5644f293067528f3b430e384aad80ca331769666b92bc57e6edbb987"
Jul 11 23:56:53.751669 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:53.751637    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v"]
Jul 11 23:56:53.868031 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:53.868003    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v"
Jul 11 23:56:53.904266 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:53.904229    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d515225-6f2e-4405-99e5-a6e755f831c8-maas-api-service-ca\") pod \"0d515225-6f2e-4405-99e5-a6e755f831c8\" (UID: \"0d515225-6f2e-4405-99e5-a6e755f831c8\") "
Jul 11 23:56:53.904576 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:53.904556    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d515225-6f2e-4405-99e5-a6e755f831c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0d515225-6f2e-4405-99e5-a6e755f831c8" (UID: "0d515225-6f2e-4405-99e5-a6e755f831c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 23:56:53.904636 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:53.904624    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d515225-6f2e-4405-99e5-a6e755f831c8-maas-api-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:56:54.734464 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:54.734415    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v"
Jul 11 23:56:54.734464 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:54.734456    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v" event={"ID":"0d515225-6f2e-4405-99e5-a6e755f831c8","Type":"ContainerDied","Data":"2437d6e446d30a4d7a449e9664bfe9c17ea576b795100a7796f762673b44a82e"}
Jul 11 23:56:54.734703 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:54.734506    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2437d6e446d30a4d7a449e9664bfe9c17ea576b795100a7796f762673b44a82e"
Jul 11 23:56:54.761575 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:54.761553    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v"]
Jul 11 23:56:54.766764 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:54.766739    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-9xs6v"]
Jul 11 23:56:54.985767 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:56:54.985679    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d515225-6f2e-4405-99e5-a6e755f831c8" path="/var/lib/kubelet/pods/0d515225-6f2e-4405-99e5-a6e755f831c8/volumes"
Jul 11 23:57:34.050551 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.050515    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn"]
Jul 11 23:57:34.050955 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.050859    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d515225-6f2e-4405-99e5-a6e755f831c8" containerName="revoke-keys"
Jul 11 23:57:34.050955 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.050870    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d515225-6f2e-4405-99e5-a6e755f831c8" containerName="revoke-keys"
Jul 11 23:57:34.050955 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.050889    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d515225-6f2e-4405-99e5-a6e755f831c8" containerName="revoke-keys"
Jul 11 23:57:34.050955 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.050895    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d515225-6f2e-4405-99e5-a6e755f831c8" containerName="revoke-keys"
Jul 11 23:57:34.050955 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.050904    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e9550d2-3722-42d3-97dd-3477c52ae66f" containerName="istio-proxy"
Jul 11 23:57:34.050955 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.050910    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e9550d2-3722-42d3-97dd-3477c52ae66f" containerName="istio-proxy"
Jul 11 23:57:34.050955 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.050917    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d515225-6f2e-4405-99e5-a6e755f831c8" containerName="revoke-keys"
Jul 11 23:57:34.050955 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.050923    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d515225-6f2e-4405-99e5-a6e755f831c8" containerName="revoke-keys"
Jul 11 23:57:34.051194 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.050974    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d515225-6f2e-4405-99e5-a6e755f831c8" containerName="revoke-keys"
Jul 11 23:57:34.051194 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.050983    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d515225-6f2e-4405-99e5-a6e755f831c8" containerName="revoke-keys"
Jul 11 23:57:34.054057 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.054040    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn"
Jul 11 23:57:34.056644 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.056621    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\""
Jul 11 23:57:34.061255 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.061229    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn"]
Jul 11 23:57:34.124017 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.123980    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"]
Jul 11 23:57:34.124267 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.124222    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk" podUID="2f87928b-cb4e-467e-8115-161e66a18c7a" containerName="istio-proxy" containerID="cri-o://cc4d562c2bd469a1d675f238bbb5402da915c1681fd72b6ba703a98393acd091" gracePeriod=30
Jul 11 23:57:34.172103 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.172070    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f4c213-6042-4db7-af2f-b85a24a91446-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn\" (UID: \"e4f4c213-6042-4db7-af2f-b85a24a91446\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn"
Jul 11 23:57:34.273129 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.273084    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f4c213-6042-4db7-af2f-b85a24a91446-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn\" (UID: \"e4f4c213-6042-4db7-af2f-b85a24a91446\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn"
Jul 11 23:57:34.273765 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.273747    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f4c213-6042-4db7-af2f-b85a24a91446-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn\" (UID: \"e4f4c213-6042-4db7-af2f-b85a24a91446\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn"
Jul 11 23:57:34.368153 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.368070    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn"
Jul 11 23:57:34.504086 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.504053    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn"]
Jul 11 23:57:34.507463 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:57:34.507404    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4f4c213_6042_4db7_af2f_b85a24a91446.slice/crio-32578ffc1789321698c188bc840332fbae94a6de2fc66918edb70245395bcb91 WatchSource:0}: Error finding container 32578ffc1789321698c188bc840332fbae94a6de2fc66918edb70245395bcb91: Status 404 returned error can't find the container with id 32578ffc1789321698c188bc840332fbae94a6de2fc66918edb70245395bcb91
Jul 11 23:57:34.920063 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.920025    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn" event={"ID":"e4f4c213-6042-4db7-af2f-b85a24a91446","Type":"ContainerStarted","Data":"fbc4b780cac88c3042869fddf8a7c19a99233f63114a3c53bd41a88b0a8799c1"}
Jul 11 23:57:34.920218 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.920080    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn" event={"ID":"e4f4c213-6042-4db7-af2f-b85a24a91446","Type":"ContainerStarted","Data":"32578ffc1789321698c188bc840332fbae94a6de2fc66918edb70245395bcb91"}
Jul 11 23:57:34.934391 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:34.934298    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn" podStartSLOduration=0.934282345 podStartE2EDuration="934.282345ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:57:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:57:34.933224214 +0000 UTC m=+1584.511316991" watchObservedRunningTime="2026-07-11 23:57:34.934282345 +0000 UTC m=+1584.512375122"
Jul 11 23:57:35.926553 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:35.926516    2569 generic.go:356] "Generic (PLEG): container finished" podID="e4f4c213-6042-4db7-af2f-b85a24a91446" containerID="fbc4b780cac88c3042869fddf8a7c19a99233f63114a3c53bd41a88b0a8799c1" exitCode=0
Jul 11 23:57:35.926924 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:35.926605    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn" event={"ID":"e4f4c213-6042-4db7-af2f-b85a24a91446","Type":"ContainerDied","Data":"fbc4b780cac88c3042869fddf8a7c19a99233f63114a3c53bd41a88b0a8799c1"}
Jul 11 23:57:37.064472 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:37.064420    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn"
Jul 11 23:57:37.100666 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:37.100637    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f4c213-6042-4db7-af2f-b85a24a91446-maas-api-service-ca\") pod \"e4f4c213-6042-4db7-af2f-b85a24a91446\" (UID: \"e4f4c213-6042-4db7-af2f-b85a24a91446\") "
Jul 11 23:57:37.100977 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:37.100953    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4f4c213-6042-4db7-af2f-b85a24a91446-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e4f4c213-6042-4db7-af2f-b85a24a91446" (UID: "e4f4c213-6042-4db7-af2f-b85a24a91446"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 23:57:37.201545 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:37.201469    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4f4c213-6042-4db7-af2f-b85a24a91446-maas-api-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:57:37.936459 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:37.936407    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn"
Jul 11 23:57:37.936459 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:37.936421    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn" event={"ID":"e4f4c213-6042-4db7-af2f-b85a24a91446","Type":"ContainerDied","Data":"32578ffc1789321698c188bc840332fbae94a6de2fc66918edb70245395bcb91"}
Jul 11 23:57:37.936660 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:37.936492    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="32578ffc1789321698c188bc840332fbae94a6de2fc66918edb70245395bcb91"
Jul 11 23:57:39.300121 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.300069    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"]
Jul 11 23:57:39.300581 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.300300    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n" podUID="8cc9f480-873b-425b-9ec6-8c25008b5a59" containerName="maas-api" containerID="cri-o://65423cb194725a2134c14ce0ce2d118c67e9b4ffc7f7c122341700b6ae304a8a" gracePeriod=30
Jul 11 23:57:39.425201 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.425180    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:57:39.523132 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.523100    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-credential-socket\") pod \"2f87928b-cb4e-467e-8115-161e66a18c7a\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") "
Jul 11 23:57:39.523313 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.523168    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-podinfo\") pod \"2f87928b-cb4e-467e-8115-161e66a18c7a\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") "
Jul 11 23:57:39.523313 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.523194    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-token\") pod \"2f87928b-cb4e-467e-8115-161e66a18c7a\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") "
Jul 11 23:57:39.523313 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.523229    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-envoy\") pod \"2f87928b-cb4e-467e-8115-161e66a18c7a\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") "
Jul 11 23:57:39.523313 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.523266    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqf4s\" (UniqueName: \"kubernetes.io/projected/2f87928b-cb4e-467e-8115-161e66a18c7a-kube-api-access-bqf4s\") pod \"2f87928b-cb4e-467e-8115-161e66a18c7a\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") "
Jul 11 23:57:39.523313 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.523295    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f87928b-cb4e-467e-8115-161e66a18c7a-istiod-ca-cert\") pod \"2f87928b-cb4e-467e-8115-161e66a18c7a\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") "
Jul 11 23:57:39.523603 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.523374    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-workload-certs\") pod \"2f87928b-cb4e-467e-8115-161e66a18c7a\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") "
Jul 11 23:57:39.523603 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.523398    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-data\") pod \"2f87928b-cb4e-467e-8115-161e66a18c7a\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") "
Jul 11 23:57:39.523603 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.523472    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-workload-socket\") pod \"2f87928b-cb4e-467e-8115-161e66a18c7a\" (UID: \"2f87928b-cb4e-467e-8115-161e66a18c7a\") "
Jul 11 23:57:39.524030 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.523985    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2f87928b-cb4e-467e-8115-161e66a18c7a" (UID: "2f87928b-cb4e-467e-8115-161e66a18c7a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 23:57:39.524030 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.524014    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2f87928b-cb4e-467e-8115-161e66a18c7a" (UID: "2f87928b-cb4e-467e-8115-161e66a18c7a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 23:57:39.524196 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.524158    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f87928b-cb4e-467e-8115-161e66a18c7a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2f87928b-cb4e-467e-8115-161e66a18c7a" (UID: "2f87928b-cb4e-467e-8115-161e66a18c7a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 23:57:39.524508 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.524463    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2f87928b-cb4e-467e-8115-161e66a18c7a" (UID: "2f87928b-cb4e-467e-8115-161e66a18c7a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 23:57:39.525034 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.524995    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-data" (OuterVolumeSpecName: "istio-data") pod "2f87928b-cb4e-467e-8115-161e66a18c7a" (UID: "2f87928b-cb4e-467e-8115-161e66a18c7a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 23:57:39.526255 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.526228    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2f87928b-cb4e-467e-8115-161e66a18c7a" (UID: "2f87928b-cb4e-467e-8115-161e66a18c7a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 23:57:39.526255 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.526242    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2f87928b-cb4e-467e-8115-161e66a18c7a" (UID: "2f87928b-cb4e-467e-8115-161e66a18c7a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 11 23:57:39.526473 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.526445    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f87928b-cb4e-467e-8115-161e66a18c7a-kube-api-access-bqf4s" (OuterVolumeSpecName: "kube-api-access-bqf4s") pod "2f87928b-cb4e-467e-8115-161e66a18c7a" (UID: "2f87928b-cb4e-467e-8115-161e66a18c7a"). InnerVolumeSpecName "kube-api-access-bqf4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 23:57:39.527465 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.527445    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-token" (OuterVolumeSpecName: "istio-token") pod "2f87928b-cb4e-467e-8115-161e66a18c7a" (UID: "2f87928b-cb4e-467e-8115-161e66a18c7a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 23:57:39.572008 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.571970    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"
Jul 11 23:57:39.624502 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.624469    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cc9f480-873b-425b-9ec6-8c25008b5a59-openshift-service-ca\") pod \"8cc9f480-873b-425b-9ec6-8c25008b5a59\" (UID: \"8cc9f480-873b-425b-9ec6-8c25008b5a59\") "
Jul 11 23:57:39.624698 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.624628    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgck7\" (UniqueName: \"kubernetes.io/projected/8cc9f480-873b-425b-9ec6-8c25008b5a59-kube-api-access-tgck7\") pod \"8cc9f480-873b-425b-9ec6-8c25008b5a59\" (UID: \"8cc9f480-873b-425b-9ec6-8c25008b5a59\") "
Jul 11 23:57:39.624698 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.624659    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cc9f480-873b-425b-9ec6-8c25008b5a59-maas-api-tls\") pod \"8cc9f480-873b-425b-9ec6-8c25008b5a59\" (UID: \"8cc9f480-873b-425b-9ec6-8c25008b5a59\") "
Jul 11 23:57:39.624909 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.624875    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cc9f480-873b-425b-9ec6-8c25008b5a59-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8cc9f480-873b-425b-9ec6-8c25008b5a59" (UID: "8cc9f480-873b-425b-9ec6-8c25008b5a59"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 23:57:39.624909 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.624892    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-workload-certs\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:57:39.624909 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.624908    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-data\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:57:39.624909 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.624917    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-workload-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:57:39.625144 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.624925    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-credential-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:57:39.625144 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.624933    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-podinfo\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:57:39.625144 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.624941    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-token\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:57:39.625144 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.624949    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f87928b-cb4e-467e-8115-161e66a18c7a-istio-envoy\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:57:39.625144 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.624956    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqf4s\" (UniqueName: \"kubernetes.io/projected/2f87928b-cb4e-467e-8115-161e66a18c7a-kube-api-access-bqf4s\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:57:39.625144 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.624965    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f87928b-cb4e-467e-8115-161e66a18c7a-istiod-ca-cert\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:57:39.626632 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.626610    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cc9f480-873b-425b-9ec6-8c25008b5a59-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8cc9f480-873b-425b-9ec6-8c25008b5a59" (UID: "8cc9f480-873b-425b-9ec6-8c25008b5a59"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 23:57:39.626743 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.626727    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cc9f480-873b-425b-9ec6-8c25008b5a59-kube-api-access-tgck7" (OuterVolumeSpecName: "kube-api-access-tgck7") pod "8cc9f480-873b-425b-9ec6-8c25008b5a59" (UID: "8cc9f480-873b-425b-9ec6-8c25008b5a59"). InnerVolumeSpecName "kube-api-access-tgck7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 23:57:39.725482 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.725355    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgck7\" (UniqueName: \"kubernetes.io/projected/8cc9f480-873b-425b-9ec6-8c25008b5a59-kube-api-access-tgck7\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:57:39.725482 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.725389    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8cc9f480-873b-425b-9ec6-8c25008b5a59-maas-api-tls\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:57:39.725482 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.725405    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8cc9f480-873b-425b-9ec6-8c25008b5a59-openshift-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:57:39.946587 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.946550    2569 generic.go:356] "Generic (PLEG): container finished" podID="8cc9f480-873b-425b-9ec6-8c25008b5a59" containerID="65423cb194725a2134c14ce0ce2d118c67e9b4ffc7f7c122341700b6ae304a8a" exitCode=0
Jul 11 23:57:39.946766 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.946595    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"
Jul 11 23:57:39.946766 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.946633    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n" event={"ID":"8cc9f480-873b-425b-9ec6-8c25008b5a59","Type":"ContainerDied","Data":"65423cb194725a2134c14ce0ce2d118c67e9b4ffc7f7c122341700b6ae304a8a"}
Jul 11 23:57:39.946766 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.946684    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n" event={"ID":"8cc9f480-873b-425b-9ec6-8c25008b5a59","Type":"ContainerDied","Data":"917a56dad8d111d367cf8ddc874faead3c065e19cb694c22e6e956d6768aba7e"}
Jul 11 23:57:39.946766 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.946706    2569 scope.go:117] "RemoveContainer" containerID="65423cb194725a2134c14ce0ce2d118c67e9b4ffc7f7c122341700b6ae304a8a"
Jul 11 23:57:39.948334 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.948309    2569 generic.go:356] "Generic (PLEG): container finished" podID="2f87928b-cb4e-467e-8115-161e66a18c7a" containerID="cc4d562c2bd469a1d675f238bbb5402da915c1681fd72b6ba703a98393acd091" exitCode=0
Jul 11 23:57:39.948473 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.948376    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"
Jul 11 23:57:39.948544 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.948379    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk" event={"ID":"2f87928b-cb4e-467e-8115-161e66a18c7a","Type":"ContainerDied","Data":"cc4d562c2bd469a1d675f238bbb5402da915c1681fd72b6ba703a98393acd091"}
Jul 11 23:57:39.948544 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.948530    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk" event={"ID":"2f87928b-cb4e-467e-8115-161e66a18c7a","Type":"ContainerDied","Data":"6db003a1baa7f08ace0de229a66fd7eeccaecba759624b6abc7d46af958c7447"}
Jul 11 23:57:39.956175 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.956151    2569 scope.go:117] "RemoveContainer" containerID="65423cb194725a2134c14ce0ce2d118c67e9b4ffc7f7c122341700b6ae304a8a"
Jul 11 23:57:39.956494 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:57:39.956465    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65423cb194725a2134c14ce0ce2d118c67e9b4ffc7f7c122341700b6ae304a8a\": container with ID starting with 65423cb194725a2134c14ce0ce2d118c67e9b4ffc7f7c122341700b6ae304a8a not found: ID does not exist" containerID="65423cb194725a2134c14ce0ce2d118c67e9b4ffc7f7c122341700b6ae304a8a"
Jul 11 23:57:39.956609 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.956506    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65423cb194725a2134c14ce0ce2d118c67e9b4ffc7f7c122341700b6ae304a8a"} err="failed to get container status \"65423cb194725a2134c14ce0ce2d118c67e9b4ffc7f7c122341700b6ae304a8a\": rpc error: code = NotFound desc = could not find container \"65423cb194725a2134c14ce0ce2d118c67e9b4ffc7f7c122341700b6ae304a8a\": container with ID starting with 65423cb194725a2134c14ce0ce2d118c67e9b4ffc7f7c122341700b6ae304a8a not found: ID does not exist"
Jul 11 23:57:39.956609 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.956525    2569 scope.go:117] "RemoveContainer" containerID="cc4d562c2bd469a1d675f238bbb5402da915c1681fd72b6ba703a98393acd091"
Jul 11 23:57:39.966818 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.966800    2569 scope.go:117] "RemoveContainer" containerID="0fb44a6e4d6bdcdccadf7d3439c6dfb178501f0b921f6925247debc45d7de0c6"
Jul 11 23:57:39.975067 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.975050    2569 scope.go:117] "RemoveContainer" containerID="cc4d562c2bd469a1d675f238bbb5402da915c1681fd72b6ba703a98393acd091"
Jul 11 23:57:39.975320 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:57:39.975300    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc4d562c2bd469a1d675f238bbb5402da915c1681fd72b6ba703a98393acd091\": container with ID starting with cc4d562c2bd469a1d675f238bbb5402da915c1681fd72b6ba703a98393acd091 not found: ID does not exist" containerID="cc4d562c2bd469a1d675f238bbb5402da915c1681fd72b6ba703a98393acd091"
Jul 11 23:57:39.975370 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.975327    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc4d562c2bd469a1d675f238bbb5402da915c1681fd72b6ba703a98393acd091"} err="failed to get container status \"cc4d562c2bd469a1d675f238bbb5402da915c1681fd72b6ba703a98393acd091\": rpc error: code = NotFound desc = could not find container \"cc4d562c2bd469a1d675f238bbb5402da915c1681fd72b6ba703a98393acd091\": container with ID starting with cc4d562c2bd469a1d675f238bbb5402da915c1681fd72b6ba703a98393acd091 not found: ID does not exist"
Jul 11 23:57:39.975370 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.975344    2569 scope.go:117] "RemoveContainer" containerID="0fb44a6e4d6bdcdccadf7d3439c6dfb178501f0b921f6925247debc45d7de0c6"
Jul 11 23:57:39.975648 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:57:39.975607    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fb44a6e4d6bdcdccadf7d3439c6dfb178501f0b921f6925247debc45d7de0c6\": container with ID starting with 0fb44a6e4d6bdcdccadf7d3439c6dfb178501f0b921f6925247debc45d7de0c6 not found: ID does not exist" containerID="0fb44a6e4d6bdcdccadf7d3439c6dfb178501f0b921f6925247debc45d7de0c6"
Jul 11 23:57:39.975648 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.975628    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fb44a6e4d6bdcdccadf7d3439c6dfb178501f0b921f6925247debc45d7de0c6"} err="failed to get container status \"0fb44a6e4d6bdcdccadf7d3439c6dfb178501f0b921f6925247debc45d7de0c6\": rpc error: code = NotFound desc = could not find container \"0fb44a6e4d6bdcdccadf7d3439c6dfb178501f0b921f6925247debc45d7de0c6\": container with ID starting with 0fb44a6e4d6bdcdccadf7d3439c6dfb178501f0b921f6925247debc45d7de0c6 not found: ID does not exist"
Jul 11 23:57:39.975851 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.975833    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"]
Jul 11 23:57:39.979149 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.979128    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-da8e2e5f-gw-openshift-default-7bc695877f-fz6qk"]
Jul 11 23:57:39.988594 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.988554    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"]
Jul 11 23:57:39.990178 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:39.990159    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-da8e2e5f-86cc56f5d5-gts7n"]
Jul 11 23:57:40.982998 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:40.982959    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f87928b-cb4e-467e-8115-161e66a18c7a" path="/var/lib/kubelet/pods/2f87928b-cb4e-467e-8115-161e66a18c7a/volumes"
Jul 11 23:57:40.983550 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:40.983530    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cc9f480-873b-425b-9ec6-8c25008b5a59" path="/var/lib/kubelet/pods/8cc9f480-873b-425b-9ec6-8c25008b5a59/volumes"
Jul 11 23:57:45.454362 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.454322    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"]
Jul 11 23:57:45.454937 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.454756    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f87928b-cb4e-467e-8115-161e66a18c7a" containerName="istio-proxy"
Jul 11 23:57:45.454937 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.454774    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f87928b-cb4e-467e-8115-161e66a18c7a" containerName="istio-proxy"
Jul 11 23:57:45.454937 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.454785    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4f4c213-6042-4db7-af2f-b85a24a91446" containerName="revoke-keys"
Jul 11 23:57:45.454937 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.454793    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4f4c213-6042-4db7-af2f-b85a24a91446" containerName="revoke-keys"
Jul 11 23:57:45.454937 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.454817    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cc9f480-873b-425b-9ec6-8c25008b5a59" containerName="maas-api"
Jul 11 23:57:45.454937 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.454825    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cc9f480-873b-425b-9ec6-8c25008b5a59" containerName="maas-api"
Jul 11 23:57:45.454937 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.454910    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f87928b-cb4e-467e-8115-161e66a18c7a" containerName="istio-proxy"
Jul 11 23:57:45.454937 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.454923    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cc9f480-873b-425b-9ec6-8c25008b5a59" containerName="maas-api"
Jul 11 23:57:45.454937 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.454932    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f87928b-cb4e-467e-8115-161e66a18c7a" containerName="istio-proxy"
Jul 11 23:57:45.454937 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.454941    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4f4c213-6042-4db7-af2f-b85a24a91446" containerName="revoke-keys"
Jul 11 23:57:45.455457 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.454950    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d515225-6f2e-4405-99e5-a6e755f831c8" containerName="revoke-keys"
Jul 11 23:57:45.457960 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.457939    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.460537 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.460516    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f85f8301-openshift-default-dockercfg-db9nn\""
Jul 11 23:57:45.470124 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.470097    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"]
Jul 11 23:57:45.586014 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.585982    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-envoy\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.586014 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.586019    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-credential-socket\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.586233 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.586037    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-podinfo\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.586233 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.586093    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-workload-certs\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.586233 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.586116    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rnq4\" (UniqueName: \"kubernetes.io/projected/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-kube-api-access-2rnq4\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.586233 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.586131    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-data\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.586233 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.586225    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-token\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.586407 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.586251    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istiod-ca-cert\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.586407 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.586287    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-workload-socket\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.687520 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.687476    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-envoy\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.687710 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.687536    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-credential-socket\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.687710 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.687560    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-podinfo\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.687710 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.687609    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-workload-certs\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.687710 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.687649    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rnq4\" (UniqueName: \"kubernetes.io/projected/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-kube-api-access-2rnq4\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.687710 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.687672    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-data\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.687970 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.687730    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-token\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.687970 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.687755    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istiod-ca-cert\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.687970 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.687789    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-workload-socket\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.687970 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.687945    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-credential-socket\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.688183 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.687997    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-workload-certs\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.688183 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.688069    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-data\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.688275 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.688178    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-workload-socket\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.688732 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.688711    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istiod-ca-cert\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.689833 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.689815    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-envoy\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.690061 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.690045    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-podinfo\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.695187 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.695165    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-token\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.695312 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.695301    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rnq4\" (UniqueName: \"kubernetes.io/projected/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-kube-api-access-2rnq4\") pod \"e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") " pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.771352 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.771258    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:45.906040 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.906014    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"]
Jul 11 23:57:45.908692 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:57:45.908664    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cefd316_54f6_4f3c_bc93_dd75466a8bcd.slice/crio-8a00ca592a10d32c644b649761f091f90eff0be0ba985597fc4e0e0625ee7c7f WatchSource:0}: Error finding container 8a00ca592a10d32c644b649761f091f90eff0be0ba985597fc4e0e0625ee7c7f: Status 404 returned error can't find the container with id 8a00ca592a10d32c644b649761f091f90eff0be0ba985597fc4e0e0625ee7c7f
Jul 11 23:57:45.910985 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.910947    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:57:45.911060 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.911032    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:57:45.911134 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.911071    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:57:45.980093 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.980062    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp" event={"ID":"2cefd316-54f6-4f3c-bc93-dd75466a8bcd","Type":"ContainerStarted","Data":"0c60a293d98ffb341aad8d9e9b4b0c10b99576f8cf17e2a48d1f4e556b23f1c8"}
Jul 11 23:57:45.980245 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.980116    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp" event={"ID":"2cefd316-54f6-4f3c-bc93-dd75466a8bcd","Type":"ContainerStarted","Data":"8a00ca592a10d32c644b649761f091f90eff0be0ba985597fc4e0e0625ee7c7f"}
Jul 11 23:57:45.999666 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:45.999588    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp" podStartSLOduration=0.999561619 podStartE2EDuration="999.561619ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:57:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:57:45.997461 +0000 UTC m=+1595.575553778" watchObservedRunningTime="2026-07-11 23:57:45.999561619 +0000 UTC m=+1595.577654402"
Jul 11 23:57:46.651003 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:46.650944    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"]
Jul 11 23:57:46.651578 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:46.651555    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f87928b-cb4e-467e-8115-161e66a18c7a" containerName="istio-proxy"
Jul 11 23:57:46.651578 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:46.651577    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f87928b-cb4e-467e-8115-161e66a18c7a" containerName="istio-proxy"
Jul 11 23:57:46.654983 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:46.654959    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"
Jul 11 23:57:46.657456 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:46.657410    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f85f8301-serving-cert\""
Jul 11 23:57:46.665459 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:46.665417    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"]
Jul 11 23:57:46.697226 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:46.697192    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a86cb80-5d38-4b2c-9aea-d26872cce65d-openshift-service-ca\") pod \"maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc\" (UID: \"4a86cb80-5d38-4b2c-9aea-d26872cce65d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"
Jul 11 23:57:46.697371 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:46.697233    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a86cb80-5d38-4b2c-9aea-d26872cce65d-maas-api-tls\") pod \"maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc\" (UID: \"4a86cb80-5d38-4b2c-9aea-d26872cce65d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"
Jul 11 23:57:46.697371 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:46.697334    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv2ng\" (UniqueName: \"kubernetes.io/projected/4a86cb80-5d38-4b2c-9aea-d26872cce65d-kube-api-access-jv2ng\") pod \"maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc\" (UID: \"4a86cb80-5d38-4b2c-9aea-d26872cce65d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"
Jul 11 23:57:46.772307 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:46.772266    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:46.780877 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:46.780847    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:46.798123 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:46.798085    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a86cb80-5d38-4b2c-9aea-d26872cce65d-openshift-service-ca\") pod \"maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc\" (UID: \"4a86cb80-5d38-4b2c-9aea-d26872cce65d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"
Jul 11 23:57:46.798295 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:46.798144    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a86cb80-5d38-4b2c-9aea-d26872cce65d-maas-api-tls\") pod \"maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc\" (UID: \"4a86cb80-5d38-4b2c-9aea-d26872cce65d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"
Jul 11 23:57:46.798295 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:46.798212    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv2ng\" (UniqueName: \"kubernetes.io/projected/4a86cb80-5d38-4b2c-9aea-d26872cce65d-kube-api-access-jv2ng\") pod \"maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc\" (UID: \"4a86cb80-5d38-4b2c-9aea-d26872cce65d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"
Jul 11 23:57:46.798382 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:57:46.798294    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-serving-cert: secret "maas-api-e2e-ait-f85f8301-serving-cert" not found
Jul 11 23:57:46.798382 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:57:46.798355    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a86cb80-5d38-4b2c-9aea-d26872cce65d-maas-api-tls podName:4a86cb80-5d38-4b2c-9aea-d26872cce65d nodeName:}" failed. No retries permitted until 2026-07-11 23:57:47.298338849 +0000 UTC m=+1596.876431603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4a86cb80-5d38-4b2c-9aea-d26872cce65d-maas-api-tls") pod "maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc" (UID: "4a86cb80-5d38-4b2c-9aea-d26872cce65d") : secret "maas-api-e2e-ait-f85f8301-serving-cert" not found
Jul 11 23:57:46.798778 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:46.798756    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a86cb80-5d38-4b2c-9aea-d26872cce65d-openshift-service-ca\") pod \"maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc\" (UID: \"4a86cb80-5d38-4b2c-9aea-d26872cce65d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"
Jul 11 23:57:46.828348 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:46.828317    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv2ng\" (UniqueName: \"kubernetes.io/projected/4a86cb80-5d38-4b2c-9aea-d26872cce65d-kube-api-access-jv2ng\") pod \"maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc\" (UID: \"4a86cb80-5d38-4b2c-9aea-d26872cce65d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"
Jul 11 23:57:46.984282 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:46.984199    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:46.985064 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:46.985045    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:47.304052 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:47.304014    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a86cb80-5d38-4b2c-9aea-d26872cce65d-maas-api-tls\") pod \"maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc\" (UID: \"4a86cb80-5d38-4b2c-9aea-d26872cce65d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"
Jul 11 23:57:47.306487 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:47.306468    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a86cb80-5d38-4b2c-9aea-d26872cce65d-maas-api-tls\") pod \"maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc\" (UID: \"4a86cb80-5d38-4b2c-9aea-d26872cce65d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"
Jul 11 23:57:47.568538 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:47.568456    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"
Jul 11 23:57:47.702683 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:47.702656    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"]
Jul 11 23:57:47.704478 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:57:47.704452    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a86cb80_5d38_4b2c_9aea_d26872cce65d.slice/crio-54de3b5816c4246f7131523a54920328f7f913b00a792990ef87fc2c7264fee9 WatchSource:0}: Error finding container 54de3b5816c4246f7131523a54920328f7f913b00a792990ef87fc2c7264fee9: Status 404 returned error can't find the container with id 54de3b5816c4246f7131523a54920328f7f913b00a792990ef87fc2c7264fee9
Jul 11 23:57:47.990690 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:47.990602    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc" event={"ID":"4a86cb80-5d38-4b2c-9aea-d26872cce65d","Type":"ContainerStarted","Data":"54de3b5816c4246f7131523a54920328f7f913b00a792990ef87fc2c7264fee9"}
Jul 11 23:57:48.996668 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:48.996568    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc" event={"ID":"4a86cb80-5d38-4b2c-9aea-d26872cce65d","Type":"ContainerStarted","Data":"59ceaa885bc1448fb09664a47e684c32b105b9cc9f3d65b7d379223f3f709a7f"}
Jul 11 23:57:49.013975 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:49.013923    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc" podStartSLOduration=2.605730276 podStartE2EDuration="3.013906874s" totalImagesPullingTime="408.176598ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:57:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:57:49.012934723 +0000 UTC m=+1598.591027523" watchObservedRunningTime="2026-07-11 23:57:49.013906874 +0000 UTC m=+1598.591999710"
Jul 11 23:57:50.002330 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:50.002291    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"
Jul 11 23:57:51.649480 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:51.649441    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj"]
Jul 11 23:57:51.653206 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:51.653182    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj"
Jul 11 23:57:51.655493 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:51.655468    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\""
Jul 11 23:57:51.659503 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:51.659477    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj"]
Jul 11 23:57:51.746193 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:51.746161    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69ff52bd-a651-4a28-82fe-18c2f9bcbb70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj\" (UID: \"69ff52bd-a651-4a28-82fe-18c2f9bcbb70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj"
Jul 11 23:57:51.847496 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:51.847457    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69ff52bd-a651-4a28-82fe-18c2f9bcbb70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj\" (UID: \"69ff52bd-a651-4a28-82fe-18c2f9bcbb70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj"
Jul 11 23:57:51.848094 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:51.848076    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69ff52bd-a651-4a28-82fe-18c2f9bcbb70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj\" (UID: \"69ff52bd-a651-4a28-82fe-18c2f9bcbb70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj"
Jul 11 23:57:51.966501 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:51.966390    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj"
Jul 11 23:57:52.137321 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:52.137298    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj"]
Jul 11 23:57:52.139544 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:57:52.139513    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69ff52bd_a651_4a28_82fe_18c2f9bcbb70.slice/crio-fd9363a78e4daa9685ecf9bd4cbd9c8c0688b1d4415bbafa90b7ec4eab13f33a WatchSource:0}: Error finding container fd9363a78e4daa9685ecf9bd4cbd9c8c0688b1d4415bbafa90b7ec4eab13f33a: Status 404 returned error can't find the container with id fd9363a78e4daa9685ecf9bd4cbd9c8c0688b1d4415bbafa90b7ec4eab13f33a
Jul 11 23:57:53.018709 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:53.018673    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj" event={"ID":"69ff52bd-a651-4a28-82fe-18c2f9bcbb70","Type":"ContainerStarted","Data":"9a1aacba3347721e04aabca3e7ee34486456a37ca2a72396b974f50a6cee2891"}
Jul 11 23:57:53.019085 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:53.018725    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj" event={"ID":"69ff52bd-a651-4a28-82fe-18c2f9bcbb70","Type":"ContainerStarted","Data":"fd9363a78e4daa9685ecf9bd4cbd9c8c0688b1d4415bbafa90b7ec4eab13f33a"}
Jul 11 23:57:53.033101 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:53.033053    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj" podStartSLOduration=2.033037694 podStartE2EDuration="2.033037694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:57:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:57:53.031993923 +0000 UTC m=+1602.610086714" watchObservedRunningTime="2026-07-11 23:57:53.033037694 +0000 UTC m=+1602.611130469"
Jul 11 23:57:54.024966 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:54.024928    2569 generic.go:356] "Generic (PLEG): container finished" podID="69ff52bd-a651-4a28-82fe-18c2f9bcbb70" containerID="9a1aacba3347721e04aabca3e7ee34486456a37ca2a72396b974f50a6cee2891" exitCode=7
Jul 11 23:57:54.025345 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:54.025008    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj" event={"ID":"69ff52bd-a651-4a28-82fe-18c2f9bcbb70","Type":"ContainerDied","Data":"9a1aacba3347721e04aabca3e7ee34486456a37ca2a72396b974f50a6cee2891"}
Jul 11 23:57:54.025345 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:54.025210    2569 scope.go:117] "RemoveContainer" containerID="9a1aacba3347721e04aabca3e7ee34486456a37ca2a72396b974f50a6cee2891"
Jul 11 23:57:55.032446 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:55.032343    2569 generic.go:356] "Generic (PLEG): container finished" podID="69ff52bd-a651-4a28-82fe-18c2f9bcbb70" containerID="5ae235bbca45e4a16c145909713168242f8cda40876b2b77e61300e4ca99cd3d" exitCode=7
Jul 11 23:57:55.032446 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:55.032377    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj" event={"ID":"69ff52bd-a651-4a28-82fe-18c2f9bcbb70","Type":"ContainerDied","Data":"5ae235bbca45e4a16c145909713168242f8cda40876b2b77e61300e4ca99cd3d"}
Jul 11 23:57:55.032446 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:55.032442    2569 scope.go:117] "RemoveContainer" containerID="9a1aacba3347721e04aabca3e7ee34486456a37ca2a72396b974f50a6cee2891"
Jul 11 23:57:55.032911 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:55.032633    2569 scope.go:117] "RemoveContainer" containerID="5ae235bbca45e4a16c145909713168242f8cda40876b2b77e61300e4ca99cd3d"
Jul 11 23:57:55.032911 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:57:55.032873    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj_odh-ai-gateway-infra(69ff52bd-a651-4a28-82fe-18c2f9bcbb70)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj" podUID="69ff52bd-a651-4a28-82fe-18c2f9bcbb70"
Jul 11 23:57:56.012626 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:56.012590    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"
Jul 11 23:57:56.039857 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:56.039827    2569 scope.go:117] "RemoveContainer" containerID="5ae235bbca45e4a16c145909713168242f8cda40876b2b77e61300e4ca99cd3d"
Jul 11 23:57:56.040214 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:57:56.040156    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj_odh-ai-gateway-infra(69ff52bd-a651-4a28-82fe-18c2f9bcbb70)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj" podUID="69ff52bd-a651-4a28-82fe-18c2f9bcbb70"
Jul 11 23:57:58.051230 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:58.051198    2569 generic.go:356] "Generic (PLEG): container finished" podID="2cefd316-54f6-4f3c-bc93-dd75466a8bcd" containerID="0c60a293d98ffb341aad8d9e9b4b0c10b99576f8cf17e2a48d1f4e556b23f1c8" exitCode=0
Jul 11 23:57:58.051751 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:58.051244    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp" event={"ID":"2cefd316-54f6-4f3c-bc93-dd75466a8bcd","Type":"ContainerDied","Data":"0c60a293d98ffb341aad8d9e9b4b0c10b99576f8cf17e2a48d1f4e556b23f1c8"}
Jul 11 23:57:58.051816 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:58.051775    2569 scope.go:117] "RemoveContainer" containerID="0c60a293d98ffb341aad8d9e9b4b0c10b99576f8cf17e2a48d1f4e556b23f1c8"
Jul 11 23:57:58.052788 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:58.052757    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:57:58.052885 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:58.052833    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:57:58.052885 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:58.052869    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:57:59.057595 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:59.057549    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp" event={"ID":"2cefd316-54f6-4f3c-bc93-dd75466a8bcd","Type":"ContainerStarted","Data":"7d11ca6f3ffd04922ed4c7ed34a922a40e33f7a18b5a86e3d8d7317e77bd6e42"}
Jul 11 23:57:59.772468 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:59.772368    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:57:59.773945 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:59.773587    2569 patch_prober.go:28] interesting pod/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 11 23:57:59.773945 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:57:59.773660    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp" podUID="2cefd316-54f6-4f3c-bc93-dd75466a8bcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 11 23:58:00.772651 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:00.772610    2569 patch_prober.go:28] interesting pod/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 11 23:58:00.773236 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:00.773200    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp" podUID="2cefd316-54f6-4f3c-bc93-dd75466a8bcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 11 23:58:01.772676 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:01.772623    2569 patch_prober.go:28] interesting pod/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 11 23:58:01.773187 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:01.772697    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp" podUID="2cefd316-54f6-4f3c-bc93-dd75466a8bcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 11 23:58:01.984575 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:01.984526    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:58:02.773153 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:02.772632    2569 patch_prober.go:28] interesting pod/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Jul 11 23:58:02.773153 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:02.772700    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp" podUID="2cefd316-54f6-4f3c-bc93-dd75466a8bcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Jul 11 23:58:03.775634 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:03.775603    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:58:03.776495 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:03.776478    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:58:08.977138 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:08.977100    2569 scope.go:117] "RemoveContainer" containerID="5ae235bbca45e4a16c145909713168242f8cda40876b2b77e61300e4ca99cd3d"
Jul 11 23:58:10.119204 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:10.119171    2569 generic.go:356] "Generic (PLEG): container finished" podID="69ff52bd-a651-4a28-82fe-18c2f9bcbb70" containerID="aa0c57ffe3edab62745690851411b7b105d1415b66bc4bb23eebe5cf934ad274" exitCode=0
Jul 11 23:58:10.119612 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:10.119236    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj" event={"ID":"69ff52bd-a651-4a28-82fe-18c2f9bcbb70","Type":"ContainerDied","Data":"aa0c57ffe3edab62745690851411b7b105d1415b66bc4bb23eebe5cf934ad274"}
Jul 11 23:58:10.119612 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:10.119301    2569 scope.go:117] "RemoveContainer" containerID="5ae235bbca45e4a16c145909713168242f8cda40876b2b77e61300e4ca99cd3d"
Jul 11 23:58:11.146867 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:11.146834    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj"]
Jul 11 23:58:11.265051 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:11.265023    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj"
Jul 11 23:58:11.431198 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:11.431096    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69ff52bd-a651-4a28-82fe-18c2f9bcbb70-maas-api-service-ca\") pod \"69ff52bd-a651-4a28-82fe-18c2f9bcbb70\" (UID: \"69ff52bd-a651-4a28-82fe-18c2f9bcbb70\") "
Jul 11 23:58:11.431511 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:11.431490    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69ff52bd-a651-4a28-82fe-18c2f9bcbb70-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "69ff52bd-a651-4a28-82fe-18c2f9bcbb70" (UID: "69ff52bd-a651-4a28-82fe-18c2f9bcbb70"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 23:58:11.532258 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:11.532223    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/69ff52bd-a651-4a28-82fe-18c2f9bcbb70-maas-api-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:58:12.130642 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:12.130615    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj"
Jul 11 23:58:12.130808 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:12.130658    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj" event={"ID":"69ff52bd-a651-4a28-82fe-18c2f9bcbb70","Type":"ContainerDied","Data":"fd9363a78e4daa9685ecf9bd4cbd9c8c0688b1d4415bbafa90b7ec4eab13f33a"}
Jul 11 23:58:12.130808 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:12.130707    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd9363a78e4daa9685ecf9bd4cbd9c8c0688b1d4415bbafa90b7ec4eab13f33a"
Jul 11 23:58:12.161043 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:12.161015    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj"]
Jul 11 23:58:12.165812 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:12.165790    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-5wlgj"]
Jul 11 23:58:12.982836 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:12.982794    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69ff52bd-a651-4a28-82fe-18c2f9bcbb70" path="/var/lib/kubelet/pods/69ff52bd-a651-4a28-82fe-18c2f9bcbb70/volumes"
Jul 11 23:58:51.685681 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:51.685643    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-mwmck"]
Jul 11 23:58:51.686149 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:51.686020    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69ff52bd-a651-4a28-82fe-18c2f9bcbb70" containerName="revoke-keys"
Jul 11 23:58:51.686149 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:51.686031    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="69ff52bd-a651-4a28-82fe-18c2f9bcbb70" containerName="revoke-keys"
Jul 11 23:58:51.686149 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:51.686043    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69ff52bd-a651-4a28-82fe-18c2f9bcbb70" containerName="revoke-keys"
Jul 11 23:58:51.686149 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:51.686048    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="69ff52bd-a651-4a28-82fe-18c2f9bcbb70" containerName="revoke-keys"
Jul 11 23:58:51.686149 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:51.686123    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="69ff52bd-a651-4a28-82fe-18c2f9bcbb70" containerName="revoke-keys"
Jul 11 23:58:51.686149 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:51.686133    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="69ff52bd-a651-4a28-82fe-18c2f9bcbb70" containerName="revoke-keys"
Jul 11 23:58:51.694818 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:51.694785    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-mwmck"
Jul 11 23:58:51.697354 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:51.697330    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-mwmck"]
Jul 11 23:58:51.697582 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:51.697453    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\""
Jul 11 23:58:51.762504 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:51.762462    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e24a802e-136d-4d7a-a230-e386c90bc5ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f85f8301-mwmck\" (UID: \"e24a802e-136d-4d7a-a230-e386c90bc5ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-mwmck"
Jul 11 23:58:51.863354 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:51.863319    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e24a802e-136d-4d7a-a230-e386c90bc5ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f85f8301-mwmck\" (UID: \"e24a802e-136d-4d7a-a230-e386c90bc5ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-mwmck"
Jul 11 23:58:51.863918 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:51.863899    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e24a802e-136d-4d7a-a230-e386c90bc5ec-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f85f8301-mwmck\" (UID: \"e24a802e-136d-4d7a-a230-e386c90bc5ec\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-mwmck"
Jul 11 23:58:52.006485 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:52.006361    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-mwmck"
Jul 11 23:58:52.137127 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:52.137101    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-mwmck"]
Jul 11 23:58:52.139916 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:58:52.139885    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode24a802e_136d_4d7a_a230_e386c90bc5ec.slice/crio-323b48fcec860f5417611d126cb4d3fb107e47898e1ae5d338818c60464a072a WatchSource:0}: Error finding container 323b48fcec860f5417611d126cb4d3fb107e47898e1ae5d338818c60464a072a: Status 404 returned error can't find the container with id 323b48fcec860f5417611d126cb4d3fb107e47898e1ae5d338818c60464a072a
Jul 11 23:58:52.325791 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:52.325757    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-mwmck" event={"ID":"e24a802e-136d-4d7a-a230-e386c90bc5ec","Type":"ContainerStarted","Data":"323b48fcec860f5417611d126cb4d3fb107e47898e1ae5d338818c60464a072a"}
Jul 11 23:58:53.331517 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:53.331484    2569 generic.go:356] "Generic (PLEG): container finished" podID="e24a802e-136d-4d7a-a230-e386c90bc5ec" containerID="2a4ee946f3717f150ed97fd358b1a1eaf91c47827e5c43388c422e4db9ecb255" exitCode=0
Jul 11 23:58:53.331996 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:53.331533    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-mwmck" event={"ID":"e24a802e-136d-4d7a-a230-e386c90bc5ec","Type":"ContainerDied","Data":"2a4ee946f3717f150ed97fd358b1a1eaf91c47827e5c43388c422e4db9ecb255"}
Jul 11 23:58:54.470504 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:54.470481    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-mwmck"
Jul 11 23:58:54.588635 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:54.588539    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e24a802e-136d-4d7a-a230-e386c90bc5ec-maas-api-service-ca\") pod \"e24a802e-136d-4d7a-a230-e386c90bc5ec\" (UID: \"e24a802e-136d-4d7a-a230-e386c90bc5ec\") "
Jul 11 23:58:54.588928 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:54.588903    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e24a802e-136d-4d7a-a230-e386c90bc5ec-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e24a802e-136d-4d7a-a230-e386c90bc5ec" (UID: "e24a802e-136d-4d7a-a230-e386c90bc5ec"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 23:58:54.690020 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:54.689984    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e24a802e-136d-4d7a-a230-e386c90bc5ec-maas-api-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:58:55.342748 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:55.342716    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-mwmck"
Jul 11 23:58:55.342748 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:55.342726    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f85f8301-mwmck" event={"ID":"e24a802e-136d-4d7a-a230-e386c90bc5ec","Type":"ContainerDied","Data":"323b48fcec860f5417611d126cb4d3fb107e47898e1ae5d338818c60464a072a"}
Jul 11 23:58:55.342958 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:58:55.342770    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="323b48fcec860f5417611d126cb4d3fb107e47898e1ae5d338818c60464a072a"
Jul 11 23:59:02.171727 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:02.171676    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"]
Jul 11 23:59:02.172182 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:02.171957    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc" podUID="4a86cb80-5d38-4b2c-9aea-d26872cce65d" containerName="maas-api" containerID="cri-o://59ceaa885bc1448fb09664a47e684c32b105b9cc9f3d65b7d379223f3f709a7f" gracePeriod=30
Jul 11 23:59:02.378300 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:02.378259    2569 generic.go:356] "Generic (PLEG): container finished" podID="4a86cb80-5d38-4b2c-9aea-d26872cce65d" containerID="59ceaa885bc1448fb09664a47e684c32b105b9cc9f3d65b7d379223f3f709a7f" exitCode=0
Jul 11 23:59:02.378517 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:02.378320    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc" event={"ID":"4a86cb80-5d38-4b2c-9aea-d26872cce65d","Type":"ContainerDied","Data":"59ceaa885bc1448fb09664a47e684c32b105b9cc9f3d65b7d379223f3f709a7f"}
Jul 11 23:59:02.427067 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:02.426998    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"
Jul 11 23:59:02.558633 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:02.558593    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a86cb80-5d38-4b2c-9aea-d26872cce65d-openshift-service-ca\") pod \"4a86cb80-5d38-4b2c-9aea-d26872cce65d\" (UID: \"4a86cb80-5d38-4b2c-9aea-d26872cce65d\") "
Jul 11 23:59:02.558807 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:02.558668    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a86cb80-5d38-4b2c-9aea-d26872cce65d-maas-api-tls\") pod \"4a86cb80-5d38-4b2c-9aea-d26872cce65d\" (UID: \"4a86cb80-5d38-4b2c-9aea-d26872cce65d\") "
Jul 11 23:59:02.558807 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:02.558737    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jv2ng\" (UniqueName: \"kubernetes.io/projected/4a86cb80-5d38-4b2c-9aea-d26872cce65d-kube-api-access-jv2ng\") pod \"4a86cb80-5d38-4b2c-9aea-d26872cce65d\" (UID: \"4a86cb80-5d38-4b2c-9aea-d26872cce65d\") "
Jul 11 23:59:02.559067 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:02.559031    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a86cb80-5d38-4b2c-9aea-d26872cce65d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4a86cb80-5d38-4b2c-9aea-d26872cce65d" (UID: "4a86cb80-5d38-4b2c-9aea-d26872cce65d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 23:59:02.560787 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:02.560755    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a86cb80-5d38-4b2c-9aea-d26872cce65d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4a86cb80-5d38-4b2c-9aea-d26872cce65d" (UID: "4a86cb80-5d38-4b2c-9aea-d26872cce65d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 11 23:59:02.560922 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:02.560856    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a86cb80-5d38-4b2c-9aea-d26872cce65d-kube-api-access-jv2ng" (OuterVolumeSpecName: "kube-api-access-jv2ng") pod "4a86cb80-5d38-4b2c-9aea-d26872cce65d" (UID: "4a86cb80-5d38-4b2c-9aea-d26872cce65d"). InnerVolumeSpecName "kube-api-access-jv2ng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 23:59:02.659473 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:02.659402    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a86cb80-5d38-4b2c-9aea-d26872cce65d-openshift-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:59:02.659473 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:02.659469    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4a86cb80-5d38-4b2c-9aea-d26872cce65d-maas-api-tls\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:59:02.659473 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:02.659480    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv2ng\" (UniqueName: \"kubernetes.io/projected/4a86cb80-5d38-4b2c-9aea-d26872cce65d-kube-api-access-jv2ng\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:59:03.384788 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:03.384758    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"
Jul 11 23:59:03.385223 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:03.384759    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc" event={"ID":"4a86cb80-5d38-4b2c-9aea-d26872cce65d","Type":"ContainerDied","Data":"54de3b5816c4246f7131523a54920328f7f913b00a792990ef87fc2c7264fee9"}
Jul 11 23:59:03.385223 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:03.384898    2569 scope.go:117] "RemoveContainer" containerID="59ceaa885bc1448fb09664a47e684c32b105b9cc9f3d65b7d379223f3f709a7f"
Jul 11 23:59:03.400793 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:03.400764    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"]
Jul 11 23:59:03.404532 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:03.404505    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f85f8301-5cc7bd8579-bhwfc"]
Jul 11 23:59:04.982746 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:04.982716    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a86cb80-5d38-4b2c-9aea-d26872cce65d" path="/var/lib/kubelet/pods/4a86cb80-5d38-4b2c-9aea-d26872cce65d/volumes"
Jul 11 23:59:17.724229 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:17.724183    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"]
Jul 11 23:59:17.724729 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:17.724474    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp" podUID="2cefd316-54f6-4f3c-bc93-dd75466a8bcd" containerName="istio-proxy" containerID="cri-o://7d11ca6f3ffd04922ed4c7ed34a922a40e33f7a18b5a86e3d8d7317e77bd6e42" gracePeriod=30
Jul 11 23:59:17.988689 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:17.988606    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"]
Jul 11 23:59:17.989058 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:17.989033    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e24a802e-136d-4d7a-a230-e386c90bc5ec" containerName="revoke-keys"
Jul 11 23:59:17.989058 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:17.989055    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e24a802e-136d-4d7a-a230-e386c90bc5ec" containerName="revoke-keys"
Jul 11 23:59:17.989209 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:17.989075    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69ff52bd-a651-4a28-82fe-18c2f9bcbb70" containerName="revoke-keys"
Jul 11 23:59:17.989209 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:17.989081    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="69ff52bd-a651-4a28-82fe-18c2f9bcbb70" containerName="revoke-keys"
Jul 11 23:59:17.989209 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:17.989123    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a86cb80-5d38-4b2c-9aea-d26872cce65d" containerName="maas-api"
Jul 11 23:59:17.989209 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:17.989133    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a86cb80-5d38-4b2c-9aea-d26872cce65d" containerName="maas-api"
Jul 11 23:59:17.989417 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:17.989224    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a86cb80-5d38-4b2c-9aea-d26872cce65d" containerName="maas-api"
Jul 11 23:59:17.989417 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:17.989240    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="69ff52bd-a651-4a28-82fe-18c2f9bcbb70" containerName="revoke-keys"
Jul 11 23:59:17.989417 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:17.989251    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e24a802e-136d-4d7a-a230-e386c90bc5ec" containerName="revoke-keys"
Jul 11 23:59:17.992316 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:17.992298    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:17.994973 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:17.994943    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-04a79e9d-openshift-default-dockercfg-6t794\""
Jul 11 23:59:18.003283 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.003249    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"]
Jul 11 23:59:18.095928 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.095714    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-workload-certs\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.095928 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.095761    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-envoy\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.095928 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.095822    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-workload-socket\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.095928 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.095846    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62399fa2-fee6-427c-b28f-44578ab32ebb-istiod-ca-cert\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.095928 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.095873    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-data\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.096331 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.095995    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-token\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.096331 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.096069    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-credential-socket\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.096331 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.096107    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-podinfo\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.096331 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.096146    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln247\" (UniqueName: \"kubernetes.io/projected/62399fa2-fee6-427c-b28f-44578ab32ebb-kube-api-access-ln247\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.197325 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.197287    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-credential-socket\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.197325 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.197331    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-podinfo\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.197609 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.197362    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln247\" (UniqueName: \"kubernetes.io/projected/62399fa2-fee6-427c-b28f-44578ab32ebb-kube-api-access-ln247\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.197609 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.197408    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-workload-certs\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.197609 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.197439    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-envoy\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.197609 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.197473    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-workload-socket\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.197609 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.197497    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62399fa2-fee6-427c-b28f-44578ab32ebb-istiod-ca-cert\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.197609 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.197525    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-data\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.197609 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.197578    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-token\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.197956 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.197811    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-credential-socket\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.197956 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.197866    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-workload-certs\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.198025 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.198004    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-data\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.198146 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.198123    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-workload-socket\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.198300 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.198281    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62399fa2-fee6-427c-b28f-44578ab32ebb-istiod-ca-cert\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.199779 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.199754    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-envoy\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.199973 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.199955    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-podinfo\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.205388 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.205363    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-token\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.205521 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.205407    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln247\" (UniqueName: \"kubernetes.io/projected/62399fa2-fee6-427c-b28f-44578ab32ebb-kube-api-access-ln247\") pod \"e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") " pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.304316 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.304280    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:18.440850 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.440815    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"]
Jul 11 23:59:18.442879 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:59:18.442852    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62399fa2_fee6_427c_b28f_44578ab32ebb.slice/crio-e1b1307e1ed2825bd8ea8cb278336415970d56629c107461049bd1a241aae2e5 WatchSource:0}: Error finding container e1b1307e1ed2825bd8ea8cb278336415970d56629c107461049bd1a241aae2e5: Status 404 returned error can't find the container with id e1b1307e1ed2825bd8ea8cb278336415970d56629c107461049bd1a241aae2e5
Jul 11 23:59:18.444960 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.444922    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:59:18.445089 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.444989    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:59:18.445089 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.445019    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:59:18.460600 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.460571    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t" event={"ID":"62399fa2-fee6-427c-b28f-44578ab32ebb","Type":"ContainerStarted","Data":"e1b1307e1ed2825bd8ea8cb278336415970d56629c107461049bd1a241aae2e5"}
Jul 11 23:59:18.778756 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.778672    2569 patch_prober.go:28] interesting pod/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 11 23:59:18.779113 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:18.778750    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp" podUID="2cefd316-54f6-4f3c-bc93-dd75466a8bcd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 11 23:59:19.211301 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:19.211264    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"]
Jul 11 23:59:19.215382 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:19.215361    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"
Jul 11 23:59:19.218727 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:19.218701    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-04a79e9d-serving-cert\""
Jul 11 23:59:19.227306 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:19.227278    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"]
Jul 11 23:59:19.309162 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:19.309120    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-maas-api-tls\") pod \"maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj\" (UID: \"663afffb-6d7f-4b64-8846-9aaeabb1bbbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"
Jul 11 23:59:19.309357 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:19.309173    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-openshift-service-ca\") pod \"maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj\" (UID: \"663afffb-6d7f-4b64-8846-9aaeabb1bbbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"
Jul 11 23:59:19.309357 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:19.309256    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5gg4\" (UniqueName: \"kubernetes.io/projected/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-kube-api-access-b5gg4\") pod \"maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj\" (UID: \"663afffb-6d7f-4b64-8846-9aaeabb1bbbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"
Jul 11 23:59:19.409766 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:19.409723    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-maas-api-tls\") pod \"maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj\" (UID: \"663afffb-6d7f-4b64-8846-9aaeabb1bbbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"
Jul 11 23:59:19.409930 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:19.409772    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-openshift-service-ca\") pod \"maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj\" (UID: \"663afffb-6d7f-4b64-8846-9aaeabb1bbbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"
Jul 11 23:59:19.409930 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:59:19.409893    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-serving-cert: secret "maas-api-e2e-derive-04a79e9d-serving-cert" not found
Jul 11 23:59:19.410066 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:19.409950    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5gg4\" (UniqueName: \"kubernetes.io/projected/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-kube-api-access-b5gg4\") pod \"maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj\" (UID: \"663afffb-6d7f-4b64-8846-9aaeabb1bbbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"
Jul 11 23:59:19.410066 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:59:19.409982    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-maas-api-tls podName:663afffb-6d7f-4b64-8846-9aaeabb1bbbf nodeName:}" failed. No retries permitted until 2026-07-11 23:59:19.909957729 +0000 UTC m=+1689.488050501 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-maas-api-tls") pod "maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj" (UID: "663afffb-6d7f-4b64-8846-9aaeabb1bbbf") : secret "maas-api-e2e-derive-04a79e9d-serving-cert" not found
Jul 11 23:59:19.410477 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:19.410455    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-openshift-service-ca\") pod \"maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj\" (UID: \"663afffb-6d7f-4b64-8846-9aaeabb1bbbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"
Jul 11 23:59:19.419260 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:19.419232    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5gg4\" (UniqueName: \"kubernetes.io/projected/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-kube-api-access-b5gg4\") pod \"maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj\" (UID: \"663afffb-6d7f-4b64-8846-9aaeabb1bbbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"
Jul 11 23:59:19.467457 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:19.467362    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t" event={"ID":"62399fa2-fee6-427c-b28f-44578ab32ebb","Type":"ContainerStarted","Data":"587ef4b2470876ea16392e4577f1b980b18e174cd4196eceae82c3050373a6fb"}
Jul 11 23:59:19.512734 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:19.512675    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t" podStartSLOduration=2.512657732 podStartE2EDuration="2.512657732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:59:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:59:19.511529436 +0000 UTC m=+1689.089622213" watchObservedRunningTime="2026-07-11 23:59:19.512657732 +0000 UTC m=+1689.090750508"
Jul 11 23:59:19.915877 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:19.915833    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-maas-api-tls\") pod \"maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj\" (UID: \"663afffb-6d7f-4b64-8846-9aaeabb1bbbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"
Jul 11 23:59:19.918214 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:19.918194    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-maas-api-tls\") pod \"maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj\" (UID: \"663afffb-6d7f-4b64-8846-9aaeabb1bbbf\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"
Jul 11 23:59:20.128003 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:20.127966    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"
Jul 11 23:59:20.267492 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:20.267456    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"]
Jul 11 23:59:20.269655 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:59:20.269624    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod663afffb_6d7f_4b64_8846_9aaeabb1bbbf.slice/crio-e769c267e0e98c2796125083cc68d99ec606a022a076b0b0f197625aa84f2afd WatchSource:0}: Error finding container e769c267e0e98c2796125083cc68d99ec606a022a076b0b0f197625aa84f2afd: Status 404 returned error can't find the container with id e769c267e0e98c2796125083cc68d99ec606a022a076b0b0f197625aa84f2afd
Jul 11 23:59:20.304954 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:20.304915    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:20.309913 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:20.309889    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:20.473997 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:20.473913    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj" event={"ID":"663afffb-6d7f-4b64-8846-9aaeabb1bbbf","Type":"ContainerStarted","Data":"e769c267e0e98c2796125083cc68d99ec606a022a076b0b0f197625aa84f2afd"}
Jul 11 23:59:20.474180 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:20.474167    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:20.475175 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:20.475155    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:21.480650 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:21.480610    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj" event={"ID":"663afffb-6d7f-4b64-8846-9aaeabb1bbbf","Type":"ContainerStarted","Data":"1379ba8f1af39bd17c6186560d25cbb77b18bc9433f1ba60c1bb6f87dfc5772a"}
Jul 11 23:59:21.499248 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:21.499201    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj" podStartSLOduration=2.028503817 podStartE2EDuration="2.499184281s" totalImagesPullingTime="470.680464ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:59:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:59:21.497841432 +0000 UTC m=+1691.075934209" watchObservedRunningTime="2026-07-11 23:59:21.499184281 +0000 UTC m=+1691.077277056"
Jul 11 23:59:22.489409 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:22.489367    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"
Jul 11 23:59:23.030600 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.030574    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:59:23.144793 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.144755    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-podinfo\") pod \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") "
Jul 11 23:59:23.144793 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.144796    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-token\") pod \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") "
Jul 11 23:59:23.145008 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.144855    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-workload-socket\") pod \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") "
Jul 11 23:59:23.145008 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.144886    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-envoy\") pod \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") "
Jul 11 23:59:23.145008 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.144900    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-credential-socket\") pod \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") "
Jul 11 23:59:23.145165 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.145079    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-workload-certs\") pod \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") "
Jul 11 23:59:23.145165 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.145126    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rnq4\" (UniqueName: \"kubernetes.io/projected/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-kube-api-access-2rnq4\") pod \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") "
Jul 11 23:59:23.145165 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.145155    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istiod-ca-cert\") pod \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") "
Jul 11 23:59:23.145320 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.145197    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-data\") pod \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\" (UID: \"2cefd316-54f6-4f3c-bc93-dd75466a8bcd\") "
Jul 11 23:59:23.145320 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.145231    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2cefd316-54f6-4f3c-bc93-dd75466a8bcd" (UID: "2cefd316-54f6-4f3c-bc93-dd75466a8bcd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 23:59:23.145320 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.145249    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2cefd316-54f6-4f3c-bc93-dd75466a8bcd" (UID: "2cefd316-54f6-4f3c-bc93-dd75466a8bcd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 23:59:23.145549 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.145528    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2cefd316-54f6-4f3c-bc93-dd75466a8bcd" (UID: "2cefd316-54f6-4f3c-bc93-dd75466a8bcd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 23:59:23.145617 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.145590    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-workload-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:59:23.145617 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.145611    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-credential-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:59:23.145723 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.145339    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2cefd316-54f6-4f3c-bc93-dd75466a8bcd" (UID: "2cefd316-54f6-4f3c-bc93-dd75466a8bcd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 23:59:23.146047 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.146025    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-data" (OuterVolumeSpecName: "istio-data") pod "2cefd316-54f6-4f3c-bc93-dd75466a8bcd" (UID: "2cefd316-54f6-4f3c-bc93-dd75466a8bcd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 23:59:23.147128 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.147106    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-token" (OuterVolumeSpecName: "istio-token") pod "2cefd316-54f6-4f3c-bc93-dd75466a8bcd" (UID: "2cefd316-54f6-4f3c-bc93-dd75466a8bcd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 23:59:23.147343 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.147322    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-kube-api-access-2rnq4" (OuterVolumeSpecName: "kube-api-access-2rnq4") pod "2cefd316-54f6-4f3c-bc93-dd75466a8bcd" (UID: "2cefd316-54f6-4f3c-bc93-dd75466a8bcd"). InnerVolumeSpecName "kube-api-access-2rnq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 11 23:59:23.147438 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.147412    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2cefd316-54f6-4f3c-bc93-dd75466a8bcd" (UID: "2cefd316-54f6-4f3c-bc93-dd75466a8bcd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 11 23:59:23.147514 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.147500    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2cefd316-54f6-4f3c-bc93-dd75466a8bcd" (UID: "2cefd316-54f6-4f3c-bc93-dd75466a8bcd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 11 23:59:23.246390 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.246350    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-podinfo\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:59:23.246390 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.246383    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-token\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:59:23.246390 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.246391    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-envoy\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:59:23.246390 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.246401    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-workload-certs\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:59:23.246688 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.246409    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rnq4\" (UniqueName: \"kubernetes.io/projected/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-kube-api-access-2rnq4\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:59:23.246688 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.246418    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istiod-ca-cert\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:59:23.246688 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.246452    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cefd316-54f6-4f3c-bc93-dd75466a8bcd-istio-data\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:59:23.494236 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.494202    2569 generic.go:356] "Generic (PLEG): container finished" podID="2cefd316-54f6-4f3c-bc93-dd75466a8bcd" containerID="7d11ca6f3ffd04922ed4c7ed34a922a40e33f7a18b5a86e3d8d7317e77bd6e42" exitCode=0
Jul 11 23:59:23.494675 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.494251    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"
Jul 11 23:59:23.494675 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.494301    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp" event={"ID":"2cefd316-54f6-4f3c-bc93-dd75466a8bcd","Type":"ContainerDied","Data":"7d11ca6f3ffd04922ed4c7ed34a922a40e33f7a18b5a86e3d8d7317e77bd6e42"}
Jul 11 23:59:23.494675 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.494370    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp" event={"ID":"2cefd316-54f6-4f3c-bc93-dd75466a8bcd","Type":"ContainerDied","Data":"8a00ca592a10d32c644b649761f091f90eff0be0ba985597fc4e0e0625ee7c7f"}
Jul 11 23:59:23.494675 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.494407    2569 scope.go:117] "RemoveContainer" containerID="7d11ca6f3ffd04922ed4c7ed34a922a40e33f7a18b5a86e3d8d7317e77bd6e42"
Jul 11 23:59:23.504384 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.504366    2569 scope.go:117] "RemoveContainer" containerID="0c60a293d98ffb341aad8d9e9b4b0c10b99576f8cf17e2a48d1f4e556b23f1c8"
Jul 11 23:59:23.512224 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.512202    2569 scope.go:117] "RemoveContainer" containerID="7d11ca6f3ffd04922ed4c7ed34a922a40e33f7a18b5a86e3d8d7317e77bd6e42"
Jul 11 23:59:23.512553 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:59:23.512530    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d11ca6f3ffd04922ed4c7ed34a922a40e33f7a18b5a86e3d8d7317e77bd6e42\": container with ID starting with 7d11ca6f3ffd04922ed4c7ed34a922a40e33f7a18b5a86e3d8d7317e77bd6e42 not found: ID does not exist" containerID="7d11ca6f3ffd04922ed4c7ed34a922a40e33f7a18b5a86e3d8d7317e77bd6e42"
Jul 11 23:59:23.512644 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.512560    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d11ca6f3ffd04922ed4c7ed34a922a40e33f7a18b5a86e3d8d7317e77bd6e42"} err="failed to get container status \"7d11ca6f3ffd04922ed4c7ed34a922a40e33f7a18b5a86e3d8d7317e77bd6e42\": rpc error: code = NotFound desc = could not find container \"7d11ca6f3ffd04922ed4c7ed34a922a40e33f7a18b5a86e3d8d7317e77bd6e42\": container with ID starting with 7d11ca6f3ffd04922ed4c7ed34a922a40e33f7a18b5a86e3d8d7317e77bd6e42 not found: ID does not exist"
Jul 11 23:59:23.512644 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.512578    2569 scope.go:117] "RemoveContainer" containerID="0c60a293d98ffb341aad8d9e9b4b0c10b99576f8cf17e2a48d1f4e556b23f1c8"
Jul 11 23:59:23.512846 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:59:23.512819    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c60a293d98ffb341aad8d9e9b4b0c10b99576f8cf17e2a48d1f4e556b23f1c8\": container with ID starting with 0c60a293d98ffb341aad8d9e9b4b0c10b99576f8cf17e2a48d1f4e556b23f1c8 not found: ID does not exist" containerID="0c60a293d98ffb341aad8d9e9b4b0c10b99576f8cf17e2a48d1f4e556b23f1c8"
Jul 11 23:59:23.512887 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.512853    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c60a293d98ffb341aad8d9e9b4b0c10b99576f8cf17e2a48d1f4e556b23f1c8"} err="failed to get container status \"0c60a293d98ffb341aad8d9e9b4b0c10b99576f8cf17e2a48d1f4e556b23f1c8\": rpc error: code = NotFound desc = could not find container \"0c60a293d98ffb341aad8d9e9b4b0c10b99576f8cf17e2a48d1f4e556b23f1c8\": container with ID starting with 0c60a293d98ffb341aad8d9e9b4b0c10b99576f8cf17e2a48d1f4e556b23f1c8 not found: ID does not exist"
Jul 11 23:59:23.516682 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.516658    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"]
Jul 11 23:59:23.522724 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.522703    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f85f8301-openshift-default-8496b8cfb5-sfwbp"]
Jul 11 23:59:23.526002 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.525981    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9"]
Jul 11 23:59:23.526333 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.526319    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cefd316-54f6-4f3c-bc93-dd75466a8bcd" containerName="istio-proxy"
Jul 11 23:59:23.526376 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.526334    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cefd316-54f6-4f3c-bc93-dd75466a8bcd" containerName="istio-proxy"
Jul 11 23:59:23.526376 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.526349    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cefd316-54f6-4f3c-bc93-dd75466a8bcd" containerName="istio-proxy"
Jul 11 23:59:23.526376 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.526355    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cefd316-54f6-4f3c-bc93-dd75466a8bcd" containerName="istio-proxy"
Jul 11 23:59:23.526483 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.526420    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cefd316-54f6-4f3c-bc93-dd75466a8bcd" containerName="istio-proxy"
Jul 11 23:59:23.531003 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.530976    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9"
Jul 11 23:59:23.535311 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.535284    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\""
Jul 11 23:59:23.536458 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.536420    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9"]
Jul 11 23:59:23.650412 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.650375    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2eb780e-74e9-46e7-b0cc-030c37c66de1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9\" (UID: \"c2eb780e-74e9-46e7-b0cc-030c37c66de1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9"
Jul 11 23:59:23.751470 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.751354    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2eb780e-74e9-46e7-b0cc-030c37c66de1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9\" (UID: \"c2eb780e-74e9-46e7-b0cc-030c37c66de1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9"
Jul 11 23:59:23.751952 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.751932    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2eb780e-74e9-46e7-b0cc-030c37c66de1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9\" (UID: \"c2eb780e-74e9-46e7-b0cc-030c37c66de1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9"
Jul 11 23:59:23.843773 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:23.843736    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9"
Jul 11 23:59:24.033160 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:24.033033    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9"]
Jul 11 23:59:24.035199 ip-10-0-136-83 kubenswrapper[2569]: W0711 23:59:24.035169    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2eb780e_74e9_46e7_b0cc_030c37c66de1.slice/crio-4a56c90ed877b5a74ef096b377a1bb7dd874680b500b17057f0551ad29824698 WatchSource:0}: Error finding container 4a56c90ed877b5a74ef096b377a1bb7dd874680b500b17057f0551ad29824698: Status 404 returned error can't find the container with id 4a56c90ed877b5a74ef096b377a1bb7dd874680b500b17057f0551ad29824698
Jul 11 23:59:24.500226 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:24.500195    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9" event={"ID":"c2eb780e-74e9-46e7-b0cc-030c37c66de1","Type":"ContainerStarted","Data":"14279ae60744fd27b38cb7ed6b6ec80621e39948baf0c2ed91269f0665323db5"}
Jul 11 23:59:24.500636 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:24.500253    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9" event={"ID":"c2eb780e-74e9-46e7-b0cc-030c37c66de1","Type":"ContainerStarted","Data":"4a56c90ed877b5a74ef096b377a1bb7dd874680b500b17057f0551ad29824698"}
Jul 11 23:59:24.516510 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:24.516456    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9" podStartSLOduration=1.516435509 podStartE2EDuration="1.516435509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:59:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:59:24.515462657 +0000 UTC m=+1694.093555434" watchObservedRunningTime="2026-07-11 23:59:24.516435509 +0000 UTC m=+1694.094528288"
Jul 11 23:59:24.986689 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:24.986648    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cefd316-54f6-4f3c-bc93-dd75466a8bcd" path="/var/lib/kubelet/pods/2cefd316-54f6-4f3c-bc93-dd75466a8bcd/volumes"
Jul 11 23:59:25.507248 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:25.507216    2569 generic.go:356] "Generic (PLEG): container finished" podID="c2eb780e-74e9-46e7-b0cc-030c37c66de1" containerID="14279ae60744fd27b38cb7ed6b6ec80621e39948baf0c2ed91269f0665323db5" exitCode=7
Jul 11 23:59:25.507678 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:25.507306    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9" event={"ID":"c2eb780e-74e9-46e7-b0cc-030c37c66de1","Type":"ContainerDied","Data":"14279ae60744fd27b38cb7ed6b6ec80621e39948baf0c2ed91269f0665323db5"}
Jul 11 23:59:25.507678 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:25.507564    2569 scope.go:117] "RemoveContainer" containerID="14279ae60744fd27b38cb7ed6b6ec80621e39948baf0c2ed91269f0665323db5"
Jul 11 23:59:26.514923 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:26.514893    2569 generic.go:356] "Generic (PLEG): container finished" podID="c2eb780e-74e9-46e7-b0cc-030c37c66de1" containerID="22304662333ffb79923f7aec7c82612c6a78980492e703b429fbf537d239a37c" exitCode=7
Jul 11 23:59:26.515329 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:26.514983    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9" event={"ID":"c2eb780e-74e9-46e7-b0cc-030c37c66de1","Type":"ContainerDied","Data":"22304662333ffb79923f7aec7c82612c6a78980492e703b429fbf537d239a37c"}
Jul 11 23:59:26.515329 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:26.515057    2569 scope.go:117] "RemoveContainer" containerID="14279ae60744fd27b38cb7ed6b6ec80621e39948baf0c2ed91269f0665323db5"
Jul 11 23:59:26.515329 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:26.515175    2569 scope.go:117] "RemoveContainer" containerID="22304662333ffb79923f7aec7c82612c6a78980492e703b429fbf537d239a37c"
Jul 11 23:59:26.515510 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:59:26.515406    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9_odh-ai-gateway-infra(c2eb780e-74e9-46e7-b0cc-030c37c66de1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9" podUID="c2eb780e-74e9-46e7-b0cc-030c37c66de1"
Jul 11 23:59:27.522828 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:27.522790    2569 scope.go:117] "RemoveContainer" containerID="22304662333ffb79923f7aec7c82612c6a78980492e703b429fbf537d239a37c"
Jul 11 23:59:27.523303 ip-10-0-136-83 kubenswrapper[2569]: E0711 23:59:27.523037    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9_odh-ai-gateway-infra(c2eb780e-74e9-46e7-b0cc-030c37c66de1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9" podUID="c2eb780e-74e9-46e7-b0cc-030c37c66de1"
Jul 11 23:59:28.500745 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:28.500713    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"
Jul 11 23:59:29.535906 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:29.535816    2569 generic.go:356] "Generic (PLEG): container finished" podID="62399fa2-fee6-427c-b28f-44578ab32ebb" containerID="587ef4b2470876ea16392e4577f1b980b18e174cd4196eceae82c3050373a6fb" exitCode=0
Jul 11 23:59:29.536341 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:29.535890    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t" event={"ID":"62399fa2-fee6-427c-b28f-44578ab32ebb","Type":"ContainerDied","Data":"587ef4b2470876ea16392e4577f1b980b18e174cd4196eceae82c3050373a6fb"}
Jul 11 23:59:29.536502 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:29.536480    2569 scope.go:117] "RemoveContainer" containerID="587ef4b2470876ea16392e4577f1b980b18e174cd4196eceae82c3050373a6fb"
Jul 11 23:59:29.537539 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:29.537511    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:59:29.537619 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:29.537582    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:59:29.537666 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:29.537619    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 11 23:59:30.543153 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:30.543118    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t" event={"ID":"62399fa2-fee6-427c-b28f-44578ab32ebb","Type":"ContainerStarted","Data":"9b7ea0f7c002f2b44503862a056eb98639b367ae9cd2c6e65dbac893264bb973"}
Jul 11 23:59:31.305211 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:31.305167    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:31.306519 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:31.306495    2569 patch_prober.go:28] interesting pod/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 11 23:59:31.306584 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:31.306545    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t" podUID="62399fa2-fee6-427c-b28f-44578ab32ebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 11 23:59:32.305341 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:32.305301    2569 patch_prober.go:28] interesting pod/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 11 23:59:32.305834 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:32.305365    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t" podUID="62399fa2-fee6-427c-b28f-44578ab32ebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 11 23:59:33.305367 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:33.305327    2569 patch_prober.go:28] interesting pod/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 11 23:59:33.305783 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:33.305385    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t" podUID="62399fa2-fee6-427c-b28f-44578ab32ebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 11 23:59:34.304811 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:34.304765    2569 patch_prober.go:28] interesting pod/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 11 23:59:34.305024 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:34.304835    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t" podUID="62399fa2-fee6-427c-b28f-44578ab32ebb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 11 23:59:35.309509 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:35.309479    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:35.309915 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:35.309807    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:35.310465 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:35.310446    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 11 23:59:40.984376 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:40.981285    2569 scope.go:117] "RemoveContainer" containerID="22304662333ffb79923f7aec7c82612c6a78980492e703b429fbf537d239a37c"
Jul 11 23:59:41.598279 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:41.598243    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9" event={"ID":"c2eb780e-74e9-46e7-b0cc-030c37c66de1","Type":"ContainerStarted","Data":"71b3b8f593e3b08bd7154793220f9286921def277dddfa83277c9315cb9b6048"}
Jul 11 23:59:42.605146 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:42.605111    2569 generic.go:356] "Generic (PLEG): container finished" podID="c2eb780e-74e9-46e7-b0cc-030c37c66de1" containerID="71b3b8f593e3b08bd7154793220f9286921def277dddfa83277c9315cb9b6048" exitCode=0
Jul 11 23:59:42.605563 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:42.605181    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9" event={"ID":"c2eb780e-74e9-46e7-b0cc-030c37c66de1","Type":"ContainerDied","Data":"71b3b8f593e3b08bd7154793220f9286921def277dddfa83277c9315cb9b6048"}
Jul 11 23:59:42.605563 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:42.605239    2569 scope.go:117] "RemoveContainer" containerID="22304662333ffb79923f7aec7c82612c6a78980492e703b429fbf537d239a37c"
Jul 11 23:59:42.628116 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:42.628085    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9"]
Jul 11 23:59:43.747904 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:43.747880    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9"
Jul 11 23:59:43.840622 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:43.840569    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2eb780e-74e9-46e7-b0cc-030c37c66de1-maas-api-service-ca\") pod \"c2eb780e-74e9-46e7-b0cc-030c37c66de1\" (UID: \"c2eb780e-74e9-46e7-b0cc-030c37c66de1\") "
Jul 11 23:59:43.840931 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:43.840907    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2eb780e-74e9-46e7-b0cc-030c37c66de1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c2eb780e-74e9-46e7-b0cc-030c37c66de1" (UID: "c2eb780e-74e9-46e7-b0cc-030c37c66de1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 11 23:59:43.941193 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:43.941104    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2eb780e-74e9-46e7-b0cc-030c37c66de1-maas-api-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 11 23:59:44.617101 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:44.617073    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9"
Jul 11 23:59:44.617101 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:44.617084    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9" event={"ID":"c2eb780e-74e9-46e7-b0cc-030c37c66de1","Type":"ContainerDied","Data":"4a56c90ed877b5a74ef096b377a1bb7dd874680b500b17057f0551ad29824698"}
Jul 11 23:59:44.617329 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:44.617137    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4a56c90ed877b5a74ef096b377a1bb7dd874680b500b17057f0551ad29824698"
Jul 11 23:59:44.646682 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:44.646647    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9"]
Jul 11 23:59:44.650085 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:44.650060    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-6pqx9"]
Jul 11 23:59:44.982626 ip-10-0-136-83 kubenswrapper[2569]: I0711 23:59:44.982546    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2eb780e-74e9-46e7-b0cc-030c37c66de1" path="/var/lib/kubelet/pods/c2eb780e-74e9-46e7-b0cc-030c37c66de1/volumes"
Jul 12 00:00:00.138204 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.138174    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-04a79e9d-29730240-27ncm"]
Jul 12 00:00:00.138696 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.138581    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2eb780e-74e9-46e7-b0cc-030c37c66de1" containerName="revoke-keys"
Jul 12 00:00:00.138696 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.138594    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2eb780e-74e9-46e7-b0cc-030c37c66de1" containerName="revoke-keys"
Jul 12 00:00:00.138696 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.138605    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2eb780e-74e9-46e7-b0cc-030c37c66de1" containerName="revoke-keys"
Jul 12 00:00:00.138696 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.138612    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2eb780e-74e9-46e7-b0cc-030c37c66de1" containerName="revoke-keys"
Jul 12 00:00:00.138696 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.138621    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2eb780e-74e9-46e7-b0cc-030c37c66de1" containerName="revoke-keys"
Jul 12 00:00:00.138696 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.138626    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2eb780e-74e9-46e7-b0cc-030c37c66de1" containerName="revoke-keys"
Jul 12 00:00:00.138696 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.138687    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2eb780e-74e9-46e7-b0cc-030c37c66de1" containerName="revoke-keys"
Jul 12 00:00:00.138696 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.138693    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2eb780e-74e9-46e7-b0cc-030c37c66de1" containerName="revoke-keys"
Jul 12 00:00:00.138696 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.138701    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cefd316-54f6-4f3c-bc93-dd75466a8bcd" containerName="istio-proxy"
Jul 12 00:00:00.138976 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.138707    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2eb780e-74e9-46e7-b0cc-030c37c66de1" containerName="revoke-keys"
Jul 12 00:00:00.142793 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.142775    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29730240-4fxqb"]
Jul 12 00:00:00.142903 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.142885    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-04a79e9d-29730240-27ncm"
Jul 12 00:00:00.145326 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.145302    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\""
Jul 12 00:00:00.146022 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.145969    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730240-4fxqb"
Jul 12 00:00:00.149636 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.149612    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29730240-4fxqb"]
Jul 12 00:00:00.151414 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.151388    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-04a79e9d-29730240-27ncm"]
Jul 12 00:00:00.178847 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.178821    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-04a79e9d-29730240-27ncm"
Jul 12 00:00:00.184961 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.184937    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730240-4fxqb"
Jul 12 00:00:00.343577 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.343399    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-04a79e9d-29730240-27ncm"]
Jul 12 00:00:00.347166 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:00:00.347129    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda329df8e_6469_4b07_b9c3_cef9fa8cea9b.slice/crio-0eb4583409881212a4d3dedf0206aba1b4d01eaaf9c3dc7c70b587ea5e3bb65f WatchSource:0}: Error finding container 0eb4583409881212a4d3dedf0206aba1b4d01eaaf9c3dc7c70b587ea5e3bb65f: Status 404 returned error can't find the container with id 0eb4583409881212a4d3dedf0206aba1b4d01eaaf9c3dc7c70b587ea5e3bb65f
Jul 12 00:00:00.370352 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.370328    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29730240-4fxqb"]
Jul 12 00:00:00.372546 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:00:00.372524    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2ae38eb_40d0_4ad4_b3cd_b59261d1e07d.slice/crio-ac3cfa832f2187030cf5c4ce0d2dd5f6168668f80408b67059d4b0fd6a9bd7de WatchSource:0}: Error finding container ac3cfa832f2187030cf5c4ce0d2dd5f6168668f80408b67059d4b0fd6a9bd7de: Status 404 returned error can't find the container with id ac3cfa832f2187030cf5c4ce0d2dd5f6168668f80408b67059d4b0fd6a9bd7de
Jul 12 00:00:00.695417 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.695382    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-04a79e9d-29730240-27ncm" event={"ID":"a329df8e-6469-4b07-b9c3-cef9fa8cea9b","Type":"ContainerStarted","Data":"e787a0b80f158cb3a8da550d1124d2fa349b3968752eb8d93068a0faf1890fee"}
Jul 12 00:00:00.695627 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.695463    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-04a79e9d-29730240-27ncm" event={"ID":"a329df8e-6469-4b07-b9c3-cef9fa8cea9b","Type":"ContainerStarted","Data":"0eb4583409881212a4d3dedf0206aba1b4d01eaaf9c3dc7c70b587ea5e3bb65f"}
Jul 12 00:00:00.697236 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.697209    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730240-4fxqb" event={"ID":"c2ae38eb-40d0-4ad4-b3cd-b59261d1e07d","Type":"ContainerStarted","Data":"6ed8564e12f7f4e178b14a2f0d7e12c1c3b904eadd79d7ef617ff912791d3ed0"}
Jul 12 00:00:00.697331 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.697258    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730240-4fxqb" event={"ID":"c2ae38eb-40d0-4ad4-b3cd-b59261d1e07d","Type":"ContainerStarted","Data":"ac3cfa832f2187030cf5c4ce0d2dd5f6168668f80408b67059d4b0fd6a9bd7de"}
Jul 12 00:00:00.711390 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.711346    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-04a79e9d-29730240-27ncm" podStartSLOduration=0.711334008 podStartE2EDuration="711.334008ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:00:00.710642057 +0000 UTC m=+1730.288734832" watchObservedRunningTime="2026-07-12 00:00:00.711334008 +0000 UTC m=+1730.289426784"
Jul 12 00:00:00.725387 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:00.725346    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730240-4fxqb" podStartSLOduration=0.725331692 podStartE2EDuration="725.331692ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:00:00.723671156 +0000 UTC m=+1730.301763931" watchObservedRunningTime="2026-07-12 00:00:00.725331692 +0000 UTC m=+1730.303424469"
Jul 12 00:00:01.704738 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:01.704707    2569 generic.go:356] "Generic (PLEG): container finished" podID="a329df8e-6469-4b07-b9c3-cef9fa8cea9b" containerID="e787a0b80f158cb3a8da550d1124d2fa349b3968752eb8d93068a0faf1890fee" exitCode=0
Jul 12 00:00:01.705197 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:01.704782    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-04a79e9d-29730240-27ncm" event={"ID":"a329df8e-6469-4b07-b9c3-cef9fa8cea9b","Type":"ContainerDied","Data":"e787a0b80f158cb3a8da550d1124d2fa349b3968752eb8d93068a0faf1890fee"}
Jul 12 00:00:01.706313 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:01.706278    2569 generic.go:356] "Generic (PLEG): container finished" podID="c2ae38eb-40d0-4ad4-b3cd-b59261d1e07d" containerID="6ed8564e12f7f4e178b14a2f0d7e12c1c3b904eadd79d7ef617ff912791d3ed0" exitCode=0
Jul 12 00:00:01.706459 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:01.706333    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730240-4fxqb" event={"ID":"c2ae38eb-40d0-4ad4-b3cd-b59261d1e07d","Type":"ContainerDied","Data":"6ed8564e12f7f4e178b14a2f0d7e12c1c3b904eadd79d7ef617ff912791d3ed0"}
Jul 12 00:00:02.855102 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:02.855076    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-04a79e9d-29730240-27ncm"
Jul 12 00:00:02.888667 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:02.888641    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730240-4fxqb"
Jul 12 00:00:03.717934 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:03.717903    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-04a79e9d-29730240-27ncm"
Jul 12 00:00:03.717934 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:03.717919    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-04a79e9d-29730240-27ncm" event={"ID":"a329df8e-6469-4b07-b9c3-cef9fa8cea9b","Type":"ContainerDied","Data":"0eb4583409881212a4d3dedf0206aba1b4d01eaaf9c3dc7c70b587ea5e3bb65f"}
Jul 12 00:00:03.718198 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:03.717971    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0eb4583409881212a4d3dedf0206aba1b4d01eaaf9c3dc7c70b587ea5e3bb65f"
Jul 12 00:00:03.719320 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:03.719290    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730240-4fxqb"
Jul 12 00:00:03.719436 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:03.719297    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730240-4fxqb" event={"ID":"c2ae38eb-40d0-4ad4-b3cd-b59261d1e07d","Type":"ContainerDied","Data":"ac3cfa832f2187030cf5c4ce0d2dd5f6168668f80408b67059d4b0fd6a9bd7de"}
Jul 12 00:00:03.719436 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:03.719415    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ac3cfa832f2187030cf5c4ce0d2dd5f6168668f80408b67059d4b0fd6a9bd7de"
Jul 12 00:00:23.564488 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:23.564450    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-qfjwn"]
Jul 12 00:00:23.565020 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:23.565003    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a329df8e-6469-4b07-b9c3-cef9fa8cea9b" containerName="cleanup"
Jul 12 00:00:23.565079 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:23.565023    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a329df8e-6469-4b07-b9c3-cef9fa8cea9b" containerName="cleanup"
Jul 12 00:00:23.565079 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:23.565039    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2ae38eb-40d0-4ad4-b3cd-b59261d1e07d" containerName="cleanup"
Jul 12 00:00:23.565079 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:23.565048    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2ae38eb-40d0-4ad4-b3cd-b59261d1e07d" containerName="cleanup"
Jul 12 00:00:23.565178 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:23.565140    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a329df8e-6469-4b07-b9c3-cef9fa8cea9b" containerName="cleanup"
Jul 12 00:00:23.565178 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:23.565157    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2ae38eb-40d0-4ad4-b3cd-b59261d1e07d" containerName="cleanup"
Jul 12 00:00:23.568716 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:23.568696    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-qfjwn"
Jul 12 00:00:23.571564 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:23.571544    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\""
Jul 12 00:00:23.573451 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:23.573413    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-qfjwn"]
Jul 12 00:00:23.700959 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:23.700915    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9df1b096-2cfa-4954-869a-c4f4a9174e31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-04a79e9d-qfjwn\" (UID: \"9df1b096-2cfa-4954-869a-c4f4a9174e31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-qfjwn"
Jul 12 00:00:23.801607 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:23.801571    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9df1b096-2cfa-4954-869a-c4f4a9174e31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-04a79e9d-qfjwn\" (UID: \"9df1b096-2cfa-4954-869a-c4f4a9174e31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-qfjwn"
Jul 12 00:00:23.802244 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:23.802214    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9df1b096-2cfa-4954-869a-c4f4a9174e31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-04a79e9d-qfjwn\" (UID: \"9df1b096-2cfa-4954-869a-c4f4a9174e31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-qfjwn"
Jul 12 00:00:23.880602 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:23.880504    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-qfjwn"
Jul 12 00:00:24.011486 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:24.011454    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-qfjwn"]
Jul 12 00:00:24.013266 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:00:24.013234    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9df1b096_2cfa_4954_869a_c4f4a9174e31.slice/crio-c28113a18712b78e293bd765745f0cc61c1468b6ad79d436ea44f230e579e9de WatchSource:0}: Error finding container c28113a18712b78e293bd765745f0cc61c1468b6ad79d436ea44f230e579e9de: Status 404 returned error can't find the container with id c28113a18712b78e293bd765745f0cc61c1468b6ad79d436ea44f230e579e9de
Jul 12 00:00:24.826966 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:24.826929    2569 generic.go:356] "Generic (PLEG): container finished" podID="9df1b096-2cfa-4954-869a-c4f4a9174e31" containerID="890564f4f95c8c3967f2522b31a4adfaa3e6bd7e8d79e47592b6d5284ba87916" exitCode=0
Jul 12 00:00:24.827393 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:24.827017    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-qfjwn" event={"ID":"9df1b096-2cfa-4954-869a-c4f4a9174e31","Type":"ContainerDied","Data":"890564f4f95c8c3967f2522b31a4adfaa3e6bd7e8d79e47592b6d5284ba87916"}
Jul 12 00:00:24.827393 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:24.827078    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-qfjwn" event={"ID":"9df1b096-2cfa-4954-869a-c4f4a9174e31","Type":"ContainerStarted","Data":"c28113a18712b78e293bd765745f0cc61c1468b6ad79d436ea44f230e579e9de"}
Jul 12 00:00:25.973822 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:25.973798    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-qfjwn"
Jul 12 00:00:26.122842 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:26.122739    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9df1b096-2cfa-4954-869a-c4f4a9174e31-maas-api-service-ca\") pod \"9df1b096-2cfa-4954-869a-c4f4a9174e31\" (UID: \"9df1b096-2cfa-4954-869a-c4f4a9174e31\") "
Jul 12 00:00:26.123143 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:26.123118    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9df1b096-2cfa-4954-869a-c4f4a9174e31-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9df1b096-2cfa-4954-869a-c4f4a9174e31" (UID: "9df1b096-2cfa-4954-869a-c4f4a9174e31"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:00:26.224301 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:26.224263    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9df1b096-2cfa-4954-869a-c4f4a9174e31-maas-api-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:00:26.844624 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:26.844588    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-qfjwn" event={"ID":"9df1b096-2cfa-4954-869a-c4f4a9174e31","Type":"ContainerDied","Data":"c28113a18712b78e293bd765745f0cc61c1468b6ad79d436ea44f230e579e9de"}
Jul 12 00:00:26.844624 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:26.844606    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-04a79e9d-qfjwn"
Jul 12 00:00:26.844895 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:26.844637    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c28113a18712b78e293bd765745f0cc61c1468b6ad79d436ea44f230e579e9de"
Jul 12 00:00:33.895253 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:33.895216    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"]
Jul 12 00:00:33.895723 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:33.895512    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj" podUID="663afffb-6d7f-4b64-8846-9aaeabb1bbbf" containerName="maas-api" containerID="cri-o://1379ba8f1af39bd17c6186560d25cbb77b18bc9433f1ba60c1bb6f87dfc5772a" gracePeriod=30
Jul 12 00:00:33.919997 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:33.919967    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-04a79e9d-29730240-27ncm"]
Jul 12 00:00:33.923657 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:33.923634    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-derive-04a79e9d-29730240-27ncm"]
Jul 12 00:00:34.153914 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.153854    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"
Jul 12 00:00:34.204652 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.204622    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-maas-api-tls\") pod \"663afffb-6d7f-4b64-8846-9aaeabb1bbbf\" (UID: \"663afffb-6d7f-4b64-8846-9aaeabb1bbbf\") "
Jul 12 00:00:34.204652 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.204654    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-openshift-service-ca\") pod \"663afffb-6d7f-4b64-8846-9aaeabb1bbbf\" (UID: \"663afffb-6d7f-4b64-8846-9aaeabb1bbbf\") "
Jul 12 00:00:34.204864 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.204698    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5gg4\" (UniqueName: \"kubernetes.io/projected/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-kube-api-access-b5gg4\") pod \"663afffb-6d7f-4b64-8846-9aaeabb1bbbf\" (UID: \"663afffb-6d7f-4b64-8846-9aaeabb1bbbf\") "
Jul 12 00:00:34.205082 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.205046    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "663afffb-6d7f-4b64-8846-9aaeabb1bbbf" (UID: "663afffb-6d7f-4b64-8846-9aaeabb1bbbf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:00:34.206781 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.206752    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "663afffb-6d7f-4b64-8846-9aaeabb1bbbf" (UID: "663afffb-6d7f-4b64-8846-9aaeabb1bbbf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 00:00:34.206900 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.206878    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-kube-api-access-b5gg4" (OuterVolumeSpecName: "kube-api-access-b5gg4") pod "663afffb-6d7f-4b64-8846-9aaeabb1bbbf" (UID: "663afffb-6d7f-4b64-8846-9aaeabb1bbbf"). InnerVolumeSpecName "kube-api-access-b5gg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:00:34.305699 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.305664    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-maas-api-tls\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:00:34.305699 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.305693    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-openshift-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:00:34.305699 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.305703    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5gg4\" (UniqueName: \"kubernetes.io/projected/663afffb-6d7f-4b64-8846-9aaeabb1bbbf-kube-api-access-b5gg4\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:00:34.885351 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.885315    2569 generic.go:356] "Generic (PLEG): container finished" podID="663afffb-6d7f-4b64-8846-9aaeabb1bbbf" containerID="1379ba8f1af39bd17c6186560d25cbb77b18bc9433f1ba60c1bb6f87dfc5772a" exitCode=0
Jul 12 00:00:34.885656 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.885361    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"
Jul 12 00:00:34.885656 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.885406    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj" event={"ID":"663afffb-6d7f-4b64-8846-9aaeabb1bbbf","Type":"ContainerDied","Data":"1379ba8f1af39bd17c6186560d25cbb77b18bc9433f1ba60c1bb6f87dfc5772a"}
Jul 12 00:00:34.885656 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.885484    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj" event={"ID":"663afffb-6d7f-4b64-8846-9aaeabb1bbbf","Type":"ContainerDied","Data":"e769c267e0e98c2796125083cc68d99ec606a022a076b0b0f197625aa84f2afd"}
Jul 12 00:00:34.885656 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.885508    2569 scope.go:117] "RemoveContainer" containerID="1379ba8f1af39bd17c6186560d25cbb77b18bc9433f1ba60c1bb6f87dfc5772a"
Jul 12 00:00:34.894326 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.894309    2569 scope.go:117] "RemoveContainer" containerID="1379ba8f1af39bd17c6186560d25cbb77b18bc9433f1ba60c1bb6f87dfc5772a"
Jul 12 00:00:34.894588 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:00:34.894566    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1379ba8f1af39bd17c6186560d25cbb77b18bc9433f1ba60c1bb6f87dfc5772a\": container with ID starting with 1379ba8f1af39bd17c6186560d25cbb77b18bc9433f1ba60c1bb6f87dfc5772a not found: ID does not exist" containerID="1379ba8f1af39bd17c6186560d25cbb77b18bc9433f1ba60c1bb6f87dfc5772a"
Jul 12 00:00:34.894687 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.894594    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1379ba8f1af39bd17c6186560d25cbb77b18bc9433f1ba60c1bb6f87dfc5772a"} err="failed to get container status \"1379ba8f1af39bd17c6186560d25cbb77b18bc9433f1ba60c1bb6f87dfc5772a\": rpc error: code = NotFound desc = could not find container \"1379ba8f1af39bd17c6186560d25cbb77b18bc9433f1ba60c1bb6f87dfc5772a\": container with ID starting with 1379ba8f1af39bd17c6186560d25cbb77b18bc9433f1ba60c1bb6f87dfc5772a not found: ID does not exist"
Jul 12 00:00:34.906018 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.905997    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"]
Jul 12 00:00:34.908919 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.908899    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-04a79e9d-785c5647c8-sppvj"]
Jul 12 00:00:34.983298 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.983254    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="663afffb-6d7f-4b64-8846-9aaeabb1bbbf" path="/var/lib/kubelet/pods/663afffb-6d7f-4b64-8846-9aaeabb1bbbf/volumes"
Jul 12 00:00:34.983661 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:34.983640    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a329df8e-6469-4b07-b9c3-cef9fa8cea9b" path="/var/lib/kubelet/pods/a329df8e-6469-4b07-b9c3-cef9fa8cea9b/volumes"
Jul 12 00:00:48.804189 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:48.804147    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"]
Jul 12 00:00:48.804618 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:48.804381    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t" podUID="62399fa2-fee6-427c-b28f-44578ab32ebb" containerName="istio-proxy" containerID="cri-o://9b7ea0f7c002f2b44503862a056eb98639b367ae9cd2c6e65dbac893264bb973" gracePeriod=30
Jul 12 00:00:50.312693 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:50.312649    2569 patch_prober.go:28] interesting pod/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 12 00:00:50.313068 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:50.312720    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t" podUID="62399fa2-fee6-427c-b28f-44578ab32ebb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 12 00:00:53.853747 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:00:53.853709    2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62399fa2_fee6_427c_b28f_44578ab32ebb.slice/crio-conmon-9b7ea0f7c002f2b44503862a056eb98639b367ae9cd2c6e65dbac893264bb973.scope\": RecentStats: unable to find data in memory cache]"
Jul 12 00:00:53.988160 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:53.988123    2569 generic.go:356] "Generic (PLEG): container finished" podID="62399fa2-fee6-427c-b28f-44578ab32ebb" containerID="9b7ea0f7c002f2b44503862a056eb98639b367ae9cd2c6e65dbac893264bb973" exitCode=0
Jul 12 00:00:53.988344 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:53.988155    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t" event={"ID":"62399fa2-fee6-427c-b28f-44578ab32ebb","Type":"ContainerDied","Data":"9b7ea0f7c002f2b44503862a056eb98639b367ae9cd2c6e65dbac893264bb973"}
Jul 12 00:00:53.988344 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:53.988229    2569 scope.go:117] "RemoveContainer" containerID="587ef4b2470876ea16392e4577f1b980b18e174cd4196eceae82c3050373a6fb"
Jul 12 00:00:54.062968 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.062910    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 12 00:00:54.185597 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.185562    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-envoy\") pod \"62399fa2-fee6-427c-b28f-44578ab32ebb\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") "
Jul 12 00:00:54.185789 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.185605    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ln247\" (UniqueName: \"kubernetes.io/projected/62399fa2-fee6-427c-b28f-44578ab32ebb-kube-api-access-ln247\") pod \"62399fa2-fee6-427c-b28f-44578ab32ebb\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") "
Jul 12 00:00:54.185789 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.185629    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-workload-certs\") pod \"62399fa2-fee6-427c-b28f-44578ab32ebb\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") "
Jul 12 00:00:54.185789 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.185703    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-workload-socket\") pod \"62399fa2-fee6-427c-b28f-44578ab32ebb\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") "
Jul 12 00:00:54.185789 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.185747    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-token\") pod \"62399fa2-fee6-427c-b28f-44578ab32ebb\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") "
Jul 12 00:00:54.185789 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.185781    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62399fa2-fee6-427c-b28f-44578ab32ebb-istiod-ca-cert\") pod \"62399fa2-fee6-427c-b28f-44578ab32ebb\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") "
Jul 12 00:00:54.186052 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.185805    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-podinfo\") pod \"62399fa2-fee6-427c-b28f-44578ab32ebb\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") "
Jul 12 00:00:54.186052 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.185840    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-credential-socket\") pod \"62399fa2-fee6-427c-b28f-44578ab32ebb\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") "
Jul 12 00:00:54.186052 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.185891    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-data\") pod \"62399fa2-fee6-427c-b28f-44578ab32ebb\" (UID: \"62399fa2-fee6-427c-b28f-44578ab32ebb\") "
Jul 12 00:00:54.186052 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.185963    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "62399fa2-fee6-427c-b28f-44578ab32ebb" (UID: "62399fa2-fee6-427c-b28f-44578ab32ebb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:00:54.186052 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.185999    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "62399fa2-fee6-427c-b28f-44578ab32ebb" (UID: "62399fa2-fee6-427c-b28f-44578ab32ebb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:00:54.186298 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.186152    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62399fa2-fee6-427c-b28f-44578ab32ebb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "62399fa2-fee6-427c-b28f-44578ab32ebb" (UID: "62399fa2-fee6-427c-b28f-44578ab32ebb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:00:54.186298 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.186253    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-workload-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:00:54.186298 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.186272    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62399fa2-fee6-427c-b28f-44578ab32ebb-istiod-ca-cert\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:00:54.186298 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.186286    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-workload-certs\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:00:54.186535 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.186352    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "62399fa2-fee6-427c-b28f-44578ab32ebb" (UID: "62399fa2-fee6-427c-b28f-44578ab32ebb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:00:54.186820 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.186778    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-data" (OuterVolumeSpecName: "istio-data") pod "62399fa2-fee6-427c-b28f-44578ab32ebb" (UID: "62399fa2-fee6-427c-b28f-44578ab32ebb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:00:54.188175 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.188112    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "62399fa2-fee6-427c-b28f-44578ab32ebb" (UID: "62399fa2-fee6-427c-b28f-44578ab32ebb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:00:54.188175 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.188112    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "62399fa2-fee6-427c-b28f-44578ab32ebb" (UID: "62399fa2-fee6-427c-b28f-44578ab32ebb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 12 00:00:54.188323 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.188250    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62399fa2-fee6-427c-b28f-44578ab32ebb-kube-api-access-ln247" (OuterVolumeSpecName: "kube-api-access-ln247") pod "62399fa2-fee6-427c-b28f-44578ab32ebb" (UID: "62399fa2-fee6-427c-b28f-44578ab32ebb"). InnerVolumeSpecName "kube-api-access-ln247". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:00:54.188388 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.188367    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-token" (OuterVolumeSpecName: "istio-token") pod "62399fa2-fee6-427c-b28f-44578ab32ebb" (UID: "62399fa2-fee6-427c-b28f-44578ab32ebb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:00:54.287286 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.287252    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-token\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:00:54.287286 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.287280    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-podinfo\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:00:54.287286 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.287291    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-credential-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:00:54.287547 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.287304    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-data\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:00:54.287547 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.287315    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62399fa2-fee6-427c-b28f-44578ab32ebb-istio-envoy\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:00:54.287547 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.287326    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ln247\" (UniqueName: \"kubernetes.io/projected/62399fa2-fee6-427c-b28f-44578ab32ebb-kube-api-access-ln247\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:00:54.993775 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.993747    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"
Jul 12 00:00:54.994081 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.993777    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t" event={"ID":"62399fa2-fee6-427c-b28f-44578ab32ebb","Type":"ContainerDied","Data":"e1b1307e1ed2825bd8ea8cb278336415970d56629c107461049bd1a241aae2e5"}
Jul 12 00:00:54.994081 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:54.993832    2569 scope.go:117] "RemoveContainer" containerID="9b7ea0f7c002f2b44503862a056eb98639b367ae9cd2c6e65dbac893264bb973"
Jul 12 00:00:55.012517 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:55.012484    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"]
Jul 12 00:00:55.016179 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:55.016156    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-04a79e9d-openshift-default-5c9cb599dc-fw55t"]
Jul 12 00:00:56.983358 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:00:56.983329    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62399fa2-fee6-427c-b28f-44578ab32ebb" path="/var/lib/kubelet/pods/62399fa2-fee6-427c-b28f-44578ab32ebb/volumes"
Jul 12 00:01:11.891770 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:01:11.891741    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 12 00:01:11.894623 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:01:11.894603    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 12 00:02:11.110826 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:11.110792    2569 scope.go:117] "RemoveContainer" containerID="545b091c239d4cfd955dcfdb08456509c17bea8e70cd17be39d0703fd573a2c9"
Jul 12 00:02:22.889104 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.889065    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"]
Jul 12 00:02:22.889595 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.889583    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62399fa2-fee6-427c-b28f-44578ab32ebb" containerName="istio-proxy"
Jul 12 00:02:22.889651 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.889599    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="62399fa2-fee6-427c-b28f-44578ab32ebb" containerName="istio-proxy"
Jul 12 00:02:22.889651 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.889623    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9df1b096-2cfa-4954-869a-c4f4a9174e31" containerName="revoke-keys"
Jul 12 00:02:22.889651 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.889632    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9df1b096-2cfa-4954-869a-c4f4a9174e31" containerName="revoke-keys"
Jul 12 00:02:22.889769 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.889654    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="663afffb-6d7f-4b64-8846-9aaeabb1bbbf" containerName="maas-api"
Jul 12 00:02:22.889769 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.889664    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="663afffb-6d7f-4b64-8846-9aaeabb1bbbf" containerName="maas-api"
Jul 12 00:02:22.889769 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.889745    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9df1b096-2cfa-4954-869a-c4f4a9174e31" containerName="revoke-keys"
Jul 12 00:02:22.889769 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.889756    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="62399fa2-fee6-427c-b28f-44578ab32ebb" containerName="istio-proxy"
Jul 12 00:02:22.889769 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.889765    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="663afffb-6d7f-4b64-8846-9aaeabb1bbbf" containerName="maas-api"
Jul 12 00:02:22.889970 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.889773    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="62399fa2-fee6-427c-b28f-44578ab32ebb" containerName="istio-proxy"
Jul 12 00:02:22.893145 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.893127    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:22.895689 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.895665    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-64f2979f-openshift-default-dockercfg-hf97x\""
Jul 12 00:02:22.903412 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.903379    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"]
Jul 12 00:02:22.966773 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.966730    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtqtc\" (UniqueName: \"kubernetes.io/projected/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-kube-api-access-rtqtc\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:22.966773 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.966772    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-data\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:22.967030 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.966809    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-workload-certs\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:22.967030 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.966845    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-workload-socket\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:22.967030 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.966860    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istiod-ca-cert\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:22.967030 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.966950    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-podinfo\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:22.967030 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.966993    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-credential-socket\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:22.967030 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.967029    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-envoy\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:22.967343 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:22.967067    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-token\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.067812 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.067774    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-token\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.068009 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.067822    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtqtc\" (UniqueName: \"kubernetes.io/projected/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-kube-api-access-rtqtc\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.068009 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.067848    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-data\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.068009 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.067882    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-workload-certs\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.068009 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.067925    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-workload-socket\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.068009 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.067947    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istiod-ca-cert\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.068009 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.067995    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-podinfo\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.068311 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.068019    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-credential-socket\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.068311 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.068061    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-envoy\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.068454 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.068329    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-workload-certs\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.068454 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.068352    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-workload-socket\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.068454 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.068412    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-data\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.068633 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.068463    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-credential-socket\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.068936 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.068916    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istiod-ca-cert\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.070493 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.070470    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-envoy\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.070646 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.070619    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-podinfo\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.075339 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.075320    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-token\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.075513 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.075497    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtqtc\" (UniqueName: \"kubernetes.io/projected/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-kube-api-access-rtqtc\") pod \"e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") " pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.207132 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.207044    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:23.349033 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.348994    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"]
Jul 12 00:02:23.350997 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:02:23.350965    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bbb7ea2_f913_4cb6_b3ec_eda2b09ed3a0.slice/crio-535fffffac27f153ca02495aac0adc772659e8a34c822bd5ff99c666d12a8d74 WatchSource:0}: Error finding container 535fffffac27f153ca02495aac0adc772659e8a34c822bd5ff99c666d12a8d74: Status 404 returned error can't find the container with id 535fffffac27f153ca02495aac0adc772659e8a34c822bd5ff99c666d12a8d74
Jul 12 00:02:23.353475 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.353414    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:02:23.353582 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.353531    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:02:23.353582 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.353575    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:02:23.440180 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.440145    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn" event={"ID":"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0","Type":"ContainerStarted","Data":"59192d3110711cf826aeb799bf606231c165dd661e7c9c11d1ec1f4b4379775c"}
Jul 12 00:02:23.440369 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.440203    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn" event={"ID":"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0","Type":"ContainerStarted","Data":"535fffffac27f153ca02495aac0adc772659e8a34c822bd5ff99c666d12a8d74"}
Jul 12 00:02:23.478407 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:23.478306    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn" podStartSLOduration=1.47828919 podStartE2EDuration="1.47828919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:02:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:02:23.47575386 +0000 UTC m=+1873.053846637" watchObservedRunningTime="2026-07-12 00:02:23.47828919 +0000 UTC m=+1873.056381965"
Jul 12 00:02:24.207940 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:24.207912    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:24.212661 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:24.212627    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:24.446370 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:24.446339    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:24.447277 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:24.447256    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:25.738846 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:25.738811    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"]
Jul 12 00:02:25.739340 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:25.739326    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62399fa2-fee6-427c-b28f-44578ab32ebb" containerName="istio-proxy"
Jul 12 00:02:25.739398 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:25.739343    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="62399fa2-fee6-427c-b28f-44578ab32ebb" containerName="istio-proxy"
Jul 12 00:02:25.742864 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:25.742843    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"
Jul 12 00:02:25.745513 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:25.745492    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-64f2979f-serving-cert\""
Jul 12 00:02:25.752909 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:25.752884    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"]
Jul 12 00:02:25.889156 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:25.889123    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/777e025a-43b4-4f4f-b370-de58b977fa1c-maas-api-tls\") pod \"maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6\" (UID: \"777e025a-43b4-4f4f-b370-de58b977fa1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"
Jul 12 00:02:25.889338 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:25.889171    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/777e025a-43b4-4f4f-b370-de58b977fa1c-openshift-service-ca\") pod \"maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6\" (UID: \"777e025a-43b4-4f4f-b370-de58b977fa1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"
Jul 12 00:02:25.889338 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:25.889227    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qswd4\" (UniqueName: \"kubernetes.io/projected/777e025a-43b4-4f4f-b370-de58b977fa1c-kube-api-access-qswd4\") pod \"maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6\" (UID: \"777e025a-43b4-4f4f-b370-de58b977fa1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"
Jul 12 00:02:25.990276 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:25.990187    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/777e025a-43b4-4f4f-b370-de58b977fa1c-openshift-service-ca\") pod \"maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6\" (UID: \"777e025a-43b4-4f4f-b370-de58b977fa1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"
Jul 12 00:02:25.990276 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:25.990245    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qswd4\" (UniqueName: \"kubernetes.io/projected/777e025a-43b4-4f4f-b370-de58b977fa1c-kube-api-access-qswd4\") pod \"maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6\" (UID: \"777e025a-43b4-4f4f-b370-de58b977fa1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"
Jul 12 00:02:25.990534 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:25.990316    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/777e025a-43b4-4f4f-b370-de58b977fa1c-maas-api-tls\") pod \"maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6\" (UID: \"777e025a-43b4-4f4f-b370-de58b977fa1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"
Jul 12 00:02:25.991021 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:25.990999    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/777e025a-43b4-4f4f-b370-de58b977fa1c-openshift-service-ca\") pod \"maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6\" (UID: \"777e025a-43b4-4f4f-b370-de58b977fa1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"
Jul 12 00:02:25.992848 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:25.992823    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/777e025a-43b4-4f4f-b370-de58b977fa1c-maas-api-tls\") pod \"maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6\" (UID: \"777e025a-43b4-4f4f-b370-de58b977fa1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"
Jul 12 00:02:26.000602 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:26.000580    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qswd4\" (UniqueName: \"kubernetes.io/projected/777e025a-43b4-4f4f-b370-de58b977fa1c-kube-api-access-qswd4\") pod \"maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6\" (UID: \"777e025a-43b4-4f4f-b370-de58b977fa1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"
Jul 12 00:02:26.055511 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:26.055474    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"
Jul 12 00:02:26.192891 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:26.192865    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"]
Jul 12 00:02:26.194571 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:02:26.194539    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod777e025a_43b4_4f4f_b370_de58b977fa1c.slice/crio-82b60e611a7626abc8b91ef47c46dba1aae76a9ed164f638928bdb3cdbc94e30 WatchSource:0}: Error finding container 82b60e611a7626abc8b91ef47c46dba1aae76a9ed164f638928bdb3cdbc94e30: Status 404 returned error can't find the container with id 82b60e611a7626abc8b91ef47c46dba1aae76a9ed164f638928bdb3cdbc94e30
Jul 12 00:02:26.195983 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:26.195966    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 12 00:02:26.457319 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:26.457282    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6" event={"ID":"777e025a-43b4-4f4f-b370-de58b977fa1c","Type":"ContainerStarted","Data":"82b60e611a7626abc8b91ef47c46dba1aae76a9ed164f638928bdb3cdbc94e30"}
Jul 12 00:02:27.463646 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:27.463613    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6" event={"ID":"777e025a-43b4-4f4f-b370-de58b977fa1c","Type":"ContainerStarted","Data":"e93bb6a9430a690aa0be71e738b2dd69a328cd37348719fe1062636fbdad7061"}
Jul 12 00:02:27.464041 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:27.463720    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"
Jul 12 00:02:27.481120 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:27.481069    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6" podStartSLOduration=1.88899574 podStartE2EDuration="2.481053487s" totalImagesPullingTime="592.057747ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:02:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:02:27.478914016 +0000 UTC m=+1877.057006805" watchObservedRunningTime="2026-07-12 00:02:27.481053487 +0000 UTC m=+1877.059146278"
Jul 12 00:02:33.481125 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:33.481083    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"
Jul 12 00:02:36.517185 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:36.517141    2569 generic.go:356] "Generic (PLEG): container finished" podID="2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" containerID="59192d3110711cf826aeb799bf606231c165dd661e7c9c11d1ec1f4b4379775c" exitCode=0
Jul 12 00:02:36.517629 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:36.517207    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn" event={"ID":"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0","Type":"ContainerDied","Data":"59192d3110711cf826aeb799bf606231c165dd661e7c9c11d1ec1f4b4379775c"}
Jul 12 00:02:36.517682 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:36.517635    2569 scope.go:117] "RemoveContainer" containerID="59192d3110711cf826aeb799bf606231c165dd661e7c9c11d1ec1f4b4379775c"
Jul 12 00:02:36.518767 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:36.518727    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:02:36.518855 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:36.518835    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:02:36.518937 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:36.518920    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:02:37.245865 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:37.245827    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb"]
Jul 12 00:02:37.248197 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:37.248178    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb"
Jul 12 00:02:37.250620 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:37.250597    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\""
Jul 12 00:02:37.256144 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:37.256119    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb"]
Jul 12 00:02:37.407171 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:37.407137    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16eb3877-fe85-4c93-abf1-e78c64a59f2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb\" (UID: \"16eb3877-fe85-4c93-abf1-e78c64a59f2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb"
Jul 12 00:02:37.507926 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:37.507836    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16eb3877-fe85-4c93-abf1-e78c64a59f2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb\" (UID: \"16eb3877-fe85-4c93-abf1-e78c64a59f2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb"
Jul 12 00:02:37.508574 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:37.508535    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16eb3877-fe85-4c93-abf1-e78c64a59f2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb\" (UID: \"16eb3877-fe85-4c93-abf1-e78c64a59f2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb"
Jul 12 00:02:37.524177 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:37.524147    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn" event={"ID":"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0","Type":"ContainerStarted","Data":"2918604af9958307118d0e7d5af3872e2acb204639845918c7149184975bb047"}
Jul 12 00:02:37.561510 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:37.561466    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb"
Jul 12 00:02:37.699159 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:37.699122    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb"]
Jul 12 00:02:37.700331 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:02:37.700306    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16eb3877_fe85_4c93_abf1_e78c64a59f2f.slice/crio-a0650d0e5b820382a6eaf6e0b44c1db4b9813901d1b13029ddc312af93307dcf WatchSource:0}: Error finding container a0650d0e5b820382a6eaf6e0b44c1db4b9813901d1b13029ddc312af93307dcf: Status 404 returned error can't find the container with id a0650d0e5b820382a6eaf6e0b44c1db4b9813901d1b13029ddc312af93307dcf
Jul 12 00:02:38.207495 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:38.207375    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:38.208356 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:38.208328    2569 patch_prober.go:28] interesting pod/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 12 00:02:38.208491 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:38.208391    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn" podUID="2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 12 00:02:38.530365 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:38.530273    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb" event={"ID":"16eb3877-fe85-4c93-abf1-e78c64a59f2f","Type":"ContainerStarted","Data":"701f0f1c5264afefab674139d4ec7a04761bda0ab34ee978e7f1c3355006668a"}
Jul 12 00:02:38.530365 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:38.530329    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb" event={"ID":"16eb3877-fe85-4c93-abf1-e78c64a59f2f","Type":"ContainerStarted","Data":"a0650d0e5b820382a6eaf6e0b44c1db4b9813901d1b13029ddc312af93307dcf"}
Jul 12 00:02:38.546241 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:38.546191    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb" podStartSLOduration=1.54617527 podStartE2EDuration="1.54617527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:02:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:02:38.544284716 +0000 UTC m=+1888.122377492" watchObservedRunningTime="2026-07-12 00:02:38.54617527 +0000 UTC m=+1888.124268045"
Jul 12 00:02:39.208293 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:39.208256    2569 patch_prober.go:28] interesting pod/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 12 00:02:39.208523 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:39.208332    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn" podUID="2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 12 00:02:39.446622 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:39.446581    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:39.538045 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:39.537955    2569 generic.go:356] "Generic (PLEG): container finished" podID="16eb3877-fe85-4c93-abf1-e78c64a59f2f" containerID="701f0f1c5264afefab674139d4ec7a04761bda0ab34ee978e7f1c3355006668a" exitCode=0
Jul 12 00:02:39.538512 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:39.538037    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb" event={"ID":"16eb3877-fe85-4c93-abf1-e78c64a59f2f","Type":"ContainerDied","Data":"701f0f1c5264afefab674139d4ec7a04761bda0ab34ee978e7f1c3355006668a"}
Jul 12 00:02:40.208269 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:40.208235    2569 patch_prober.go:28] interesting pod/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 12 00:02:40.208453 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:40.208292    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn" podUID="2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 12 00:02:40.710507 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:40.710482    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb"
Jul 12 00:02:40.736383 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:40.736351    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16eb3877-fe85-4c93-abf1-e78c64a59f2f-maas-api-service-ca\") pod \"16eb3877-fe85-4c93-abf1-e78c64a59f2f\" (UID: \"16eb3877-fe85-4c93-abf1-e78c64a59f2f\") "
Jul 12 00:02:40.736808 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:40.736780    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16eb3877-fe85-4c93-abf1-e78c64a59f2f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "16eb3877-fe85-4c93-abf1-e78c64a59f2f" (UID: "16eb3877-fe85-4c93-abf1-e78c64a59f2f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:02:40.837230 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:40.837124    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16eb3877-fe85-4c93-abf1-e78c64a59f2f-maas-api-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:02:41.207694 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:41.207606    2569 patch_prober.go:28] interesting pod/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 12 00:02:41.207694 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:41.207665    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn" podUID="2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 12 00:02:41.548721 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:41.548693    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb"
Jul 12 00:02:41.548905 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:41.548689    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb" event={"ID":"16eb3877-fe85-4c93-abf1-e78c64a59f2f","Type":"ContainerDied","Data":"a0650d0e5b820382a6eaf6e0b44c1db4b9813901d1b13029ddc312af93307dcf"}
Jul 12 00:02:41.548905 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:41.548832    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0650d0e5b820382a6eaf6e0b44c1db4b9813901d1b13029ddc312af93307dcf"
Jul 12 00:02:42.211603 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:42.211573    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:42.212533 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:42.212514    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:02:47.602339 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:47.602303    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"]
Jul 12 00:02:47.603328 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:47.603289    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6" podUID="777e025a-43b4-4f4f-b370-de58b977fa1c" containerName="maas-api" containerID="cri-o://e93bb6a9430a690aa0be71e738b2dd69a328cd37348719fe1062636fbdad7061" gracePeriod=30
Jul 12 00:02:47.876320 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:47.876291    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"
Jul 12 00:02:47.898499 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:47.898468    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/777e025a-43b4-4f4f-b370-de58b977fa1c-openshift-service-ca\") pod \"777e025a-43b4-4f4f-b370-de58b977fa1c\" (UID: \"777e025a-43b4-4f4f-b370-de58b977fa1c\") "
Jul 12 00:02:47.898673 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:47.898605    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/777e025a-43b4-4f4f-b370-de58b977fa1c-maas-api-tls\") pod \"777e025a-43b4-4f4f-b370-de58b977fa1c\" (UID: \"777e025a-43b4-4f4f-b370-de58b977fa1c\") "
Jul 12 00:02:47.898673 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:47.898652    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qswd4\" (UniqueName: \"kubernetes.io/projected/777e025a-43b4-4f4f-b370-de58b977fa1c-kube-api-access-qswd4\") pod \"777e025a-43b4-4f4f-b370-de58b977fa1c\" (UID: \"777e025a-43b4-4f4f-b370-de58b977fa1c\") "
Jul 12 00:02:47.898889 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:47.898864    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/777e025a-43b4-4f4f-b370-de58b977fa1c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "777e025a-43b4-4f4f-b370-de58b977fa1c" (UID: "777e025a-43b4-4f4f-b370-de58b977fa1c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:02:47.900738 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:47.900712    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/777e025a-43b4-4f4f-b370-de58b977fa1c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "777e025a-43b4-4f4f-b370-de58b977fa1c" (UID: "777e025a-43b4-4f4f-b370-de58b977fa1c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 00:02:47.900738 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:47.900727    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/777e025a-43b4-4f4f-b370-de58b977fa1c-kube-api-access-qswd4" (OuterVolumeSpecName: "kube-api-access-qswd4") pod "777e025a-43b4-4f4f-b370-de58b977fa1c" (UID: "777e025a-43b4-4f4f-b370-de58b977fa1c"). InnerVolumeSpecName "kube-api-access-qswd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:02:48.000067 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:48.000025    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qswd4\" (UniqueName: \"kubernetes.io/projected/777e025a-43b4-4f4f-b370-de58b977fa1c-kube-api-access-qswd4\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:02:48.000067 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:48.000057    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/777e025a-43b4-4f4f-b370-de58b977fa1c-openshift-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:02:48.000067 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:48.000071    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/777e025a-43b4-4f4f-b370-de58b977fa1c-maas-api-tls\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:02:48.584500 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:48.584463    2569 generic.go:356] "Generic (PLEG): container finished" podID="777e025a-43b4-4f4f-b370-de58b977fa1c" containerID="e93bb6a9430a690aa0be71e738b2dd69a328cd37348719fe1062636fbdad7061" exitCode=0
Jul 12 00:02:48.584668 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:48.584513    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"
Jul 12 00:02:48.584668 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:48.584528    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6" event={"ID":"777e025a-43b4-4f4f-b370-de58b977fa1c","Type":"ContainerDied","Data":"e93bb6a9430a690aa0be71e738b2dd69a328cd37348719fe1062636fbdad7061"}
Jul 12 00:02:48.584668 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:48.584582    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6" event={"ID":"777e025a-43b4-4f4f-b370-de58b977fa1c","Type":"ContainerDied","Data":"82b60e611a7626abc8b91ef47c46dba1aae76a9ed164f638928bdb3cdbc94e30"}
Jul 12 00:02:48.584668 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:48.584611    2569 scope.go:117] "RemoveContainer" containerID="e93bb6a9430a690aa0be71e738b2dd69a328cd37348719fe1062636fbdad7061"
Jul 12 00:02:48.593527 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:48.593510    2569 scope.go:117] "RemoveContainer" containerID="e93bb6a9430a690aa0be71e738b2dd69a328cd37348719fe1062636fbdad7061"
Jul 12 00:02:48.593793 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:02:48.593774    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e93bb6a9430a690aa0be71e738b2dd69a328cd37348719fe1062636fbdad7061\": container with ID starting with e93bb6a9430a690aa0be71e738b2dd69a328cd37348719fe1062636fbdad7061 not found: ID does not exist" containerID="e93bb6a9430a690aa0be71e738b2dd69a328cd37348719fe1062636fbdad7061"
Jul 12 00:02:48.593854 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:48.593802    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93bb6a9430a690aa0be71e738b2dd69a328cd37348719fe1062636fbdad7061"} err="failed to get container status \"e93bb6a9430a690aa0be71e738b2dd69a328cd37348719fe1062636fbdad7061\": rpc error: code = NotFound desc = could not find container \"e93bb6a9430a690aa0be71e738b2dd69a328cd37348719fe1062636fbdad7061\": container with ID starting with e93bb6a9430a690aa0be71e738b2dd69a328cd37348719fe1062636fbdad7061 not found: ID does not exist"
Jul 12 00:02:48.605316 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:48.605283    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"]
Jul 12 00:02:48.608632 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:48.608605    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-64f2979f-d4659c4b6-22qh6"]
Jul 12 00:02:48.983269 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:02:48.983181    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="777e025a-43b4-4f4f-b370-de58b977fa1c" path="/var/lib/kubelet/pods/777e025a-43b4-4f4f-b370-de58b977fa1c/volumes"
Jul 12 00:03:02.966071 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:02.966024    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"]
Jul 12 00:03:02.966674 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:02.966262    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn" podUID="2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" containerName="istio-proxy" containerID="cri-o://2918604af9958307118d0e7d5af3872e2acb204639845918c7149184975bb047" gracePeriod=30
Jul 12 00:03:08.220791 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.220764    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:03:08.386353 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.386319    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-workload-socket\") pod \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") "
Jul 12 00:03:08.386353 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.386358    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-podinfo\") pod \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") "
Jul 12 00:03:08.386640 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.386378    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtqtc\" (UniqueName: \"kubernetes.io/projected/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-kube-api-access-rtqtc\") pod \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") "
Jul 12 00:03:08.386640 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.386411    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-credential-socket\") pod \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") "
Jul 12 00:03:08.386640 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.386456    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-token\") pod \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") "
Jul 12 00:03:08.386640 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.386482    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istiod-ca-cert\") pod \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") "
Jul 12 00:03:08.386640 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.386507    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-envoy\") pod \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") "
Jul 12 00:03:08.386640 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.386564    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-data\") pod \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") "
Jul 12 00:03:08.386640 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.386590    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-workload-certs\") pod \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\" (UID: \"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0\") "
Jul 12 00:03:08.387013 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.386708    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" (UID: "2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:03:08.387013 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.386909    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-workload-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:03:08.387013 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.386906    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" (UID: "2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:03:08.387013 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.386932    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" (UID: "2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:03:08.387013 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.386991    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" (UID: "2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:03:08.387917 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.387888    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-data" (OuterVolumeSpecName: "istio-data") pod "2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" (UID: "2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:03:08.388714 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.388688    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" (UID: "2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 12 00:03:08.388796 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.388710    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-kube-api-access-rtqtc" (OuterVolumeSpecName: "kube-api-access-rtqtc") pod "2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" (UID: "2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0"). InnerVolumeSpecName "kube-api-access-rtqtc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:03:08.388796 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.388780    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-token" (OuterVolumeSpecName: "istio-token") pod "2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" (UID: "2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:03:08.388889 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.388802    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" (UID: "2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:03:08.487738 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.487687    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-podinfo\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:03:08.487738 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.487731    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtqtc\" (UniqueName: \"kubernetes.io/projected/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-kube-api-access-rtqtc\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:03:08.487738 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.487744    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-credential-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:03:08.487989 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.487757    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-token\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:03:08.487989 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.487769    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istiod-ca-cert\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:03:08.487989 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.487779    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-envoy\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:03:08.487989 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.487789    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-istio-data\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:03:08.487989 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.487799    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0-workload-certs\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:03:08.688446 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.688337    2569 generic.go:356] "Generic (PLEG): container finished" podID="2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" containerID="2918604af9958307118d0e7d5af3872e2acb204639845918c7149184975bb047" exitCode=0
Jul 12 00:03:08.688446 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.688382    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"
Jul 12 00:03:08.688446 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.688418    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn" event={"ID":"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0","Type":"ContainerDied","Data":"2918604af9958307118d0e7d5af3872e2acb204639845918c7149184975bb047"}
Jul 12 00:03:08.688691 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.688493    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn" event={"ID":"2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0","Type":"ContainerDied","Data":"535fffffac27f153ca02495aac0adc772659e8a34c822bd5ff99c666d12a8d74"}
Jul 12 00:03:08.688691 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.688516    2569 scope.go:117] "RemoveContainer" containerID="2918604af9958307118d0e7d5af3872e2acb204639845918c7149184975bb047"
Jul 12 00:03:08.697743 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.697729    2569 scope.go:117] "RemoveContainer" containerID="59192d3110711cf826aeb799bf606231c165dd661e7c9c11d1ec1f4b4379775c"
Jul 12 00:03:08.705293 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.705276    2569 scope.go:117] "RemoveContainer" containerID="2918604af9958307118d0e7d5af3872e2acb204639845918c7149184975bb047"
Jul 12 00:03:08.705545 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:03:08.705528    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2918604af9958307118d0e7d5af3872e2acb204639845918c7149184975bb047\": container with ID starting with 2918604af9958307118d0e7d5af3872e2acb204639845918c7149184975bb047 not found: ID does not exist" containerID="2918604af9958307118d0e7d5af3872e2acb204639845918c7149184975bb047"
Jul 12 00:03:08.705598 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.705553    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2918604af9958307118d0e7d5af3872e2acb204639845918c7149184975bb047"} err="failed to get container status \"2918604af9958307118d0e7d5af3872e2acb204639845918c7149184975bb047\": rpc error: code = NotFound desc = could not find container \"2918604af9958307118d0e7d5af3872e2acb204639845918c7149184975bb047\": container with ID starting with 2918604af9958307118d0e7d5af3872e2acb204639845918c7149184975bb047 not found: ID does not exist"
Jul 12 00:03:08.705598 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.705569    2569 scope.go:117] "RemoveContainer" containerID="59192d3110711cf826aeb799bf606231c165dd661e7c9c11d1ec1f4b4379775c"
Jul 12 00:03:08.705801 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:03:08.705783    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59192d3110711cf826aeb799bf606231c165dd661e7c9c11d1ec1f4b4379775c\": container with ID starting with 59192d3110711cf826aeb799bf606231c165dd661e7c9c11d1ec1f4b4379775c not found: ID does not exist" containerID="59192d3110711cf826aeb799bf606231c165dd661e7c9c11d1ec1f4b4379775c"
Jul 12 00:03:08.705838 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.705806    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59192d3110711cf826aeb799bf606231c165dd661e7c9c11d1ec1f4b4379775c"} err="failed to get container status \"59192d3110711cf826aeb799bf606231c165dd661e7c9c11d1ec1f4b4379775c\": rpc error: code = NotFound desc = could not find container \"59192d3110711cf826aeb799bf606231c165dd661e7c9c11d1ec1f4b4379775c\": container with ID starting with 59192d3110711cf826aeb799bf606231c165dd661e7c9c11d1ec1f4b4379775c not found: ID does not exist"
Jul 12 00:03:08.709607 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.709586    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"]
Jul 12 00:03:08.715077 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.715058    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-64f2979f-openshift-default-79bffb4dc9-6d4vn"]
Jul 12 00:03:08.982802 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:08.982706    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" path="/var/lib/kubelet/pods/2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0/volumes"
Jul 12 00:03:11.131504 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:11.131471    2569 scope.go:117] "RemoveContainer" containerID="097cb96ead74cf4b44765fcb4f665eb9d56f5b5866f783c717acdb439fe598c1"
Jul 12 00:03:22.234671 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.234640    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"]
Jul 12 00:03:22.235184 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.234988    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" containerName="istio-proxy"
Jul 12 00:03:22.235184 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.235000    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" containerName="istio-proxy"
Jul 12 00:03:22.235184 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.235020    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16eb3877-fe85-4c93-abf1-e78c64a59f2f" containerName="revoke-keys"
Jul 12 00:03:22.235184 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.235025    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="16eb3877-fe85-4c93-abf1-e78c64a59f2f" containerName="revoke-keys"
Jul 12 00:03:22.235184 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.235035    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="777e025a-43b4-4f4f-b370-de58b977fa1c" containerName="maas-api"
Jul 12 00:03:22.235184 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.235040    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="777e025a-43b4-4f4f-b370-de58b977fa1c" containerName="maas-api"
Jul 12 00:03:22.235184 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.235091    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" containerName="istio-proxy"
Jul 12 00:03:22.235184 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.235099    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="16eb3877-fe85-4c93-abf1-e78c64a59f2f" containerName="revoke-keys"
Jul 12 00:03:22.235184 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.235109    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="777e025a-43b4-4f4f-b370-de58b977fa1c" containerName="maas-api"
Jul 12 00:03:22.238223 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.238200    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.240673 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.240650    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-14eebdc1-openshift-default-dockercfg-l7qmd\""
Jul 12 00:03:22.247865 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.247836    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"]
Jul 12 00:03:22.410488 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.410420    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-podinfo\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.410488 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.410489    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-envoy\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.410735 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.410568    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ktqd\" (UniqueName: \"kubernetes.io/projected/71c29536-da15-43ce-947a-fa8f37d48cc5-kube-api-access-6ktqd\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.410735 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.410610    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-data\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.410735 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.410694    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-workload-socket\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.410876 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.410740    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-token\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.410876 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.410763    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-credential-socket\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.410876 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.410783    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71c29536-da15-43ce-947a-fa8f37d48cc5-istiod-ca-cert\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.410876 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.410858    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-workload-certs\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.512350 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.512255    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-workload-socket\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.512350 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.512307    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-token\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.512350 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.512329    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-credential-socket\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.512350 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.512350    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71c29536-da15-43ce-947a-fa8f37d48cc5-istiod-ca-cert\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.512750 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.512370    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-workload-certs\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.512750 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.512413    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-podinfo\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.512750 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.512470    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-envoy\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.512750 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.512504    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ktqd\" (UniqueName: \"kubernetes.io/projected/71c29536-da15-43ce-947a-fa8f37d48cc5-kube-api-access-6ktqd\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.512750 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.512534    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-data\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.513025 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.512766    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-credential-socket\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.513025 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.512790    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-workload-socket\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.513025 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.512866    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-data\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.513025 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.512897    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-workload-certs\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.513200 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.513163    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71c29536-da15-43ce-947a-fa8f37d48cc5-istiod-ca-cert\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.514801 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.514775    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-envoy\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.515018 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.515000    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-podinfo\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.520085 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.520062    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-token\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.520290 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.520271    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ktqd\" (UniqueName: \"kubernetes.io/projected/71c29536-da15-43ce-947a-fa8f37d48cc5-kube-api-access-6ktqd\") pod \"e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") " pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.549223 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.549184    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:22.689364 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.689335    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"]
Jul 12 00:03:22.691459 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:03:22.691417    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71c29536_da15_43ce_947a_fa8f37d48cc5.slice/crio-0a66db8631d31ca47ae1f5f7c329ac6315654d93512f6324bb1d0974826925b6 WatchSource:0}: Error finding container 0a66db8631d31ca47ae1f5f7c329ac6315654d93512f6324bb1d0974826925b6: Status 404 returned error can't find the container with id 0a66db8631d31ca47ae1f5f7c329ac6315654d93512f6324bb1d0974826925b6
Jul 12 00:03:22.694021 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.693989    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:03:22.694092 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.694060    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:03:22.694131 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.694093    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:03:22.766404 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.766337    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7" event={"ID":"71c29536-da15-43ce-947a-fa8f37d48cc5","Type":"ContainerStarted","Data":"4d8f54e77664592b5453a30c870509af4d04255c69500ca64b7bd614ce20b99e"}
Jul 12 00:03:22.766404 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.766386    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7" event={"ID":"71c29536-da15-43ce-947a-fa8f37d48cc5","Type":"ContainerStarted","Data":"0a66db8631d31ca47ae1f5f7c329ac6315654d93512f6324bb1d0974826925b6"}
Jul 12 00:03:22.786879 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:22.786814    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7" podStartSLOduration=0.78679373 podStartE2EDuration="786.79373ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:03:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:03:22.785571638 +0000 UTC m=+1932.363664414" watchObservedRunningTime="2026-07-12 00:03:22.78679373 +0000 UTC m=+1932.364886508"
Jul 12 00:03:23.549636 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:23.549601    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:23.555034 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:23.555005    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:23.771299 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:23.771268    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:23.772138 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:23.772118    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:24.709505 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:24.709471    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"]
Jul 12 00:03:24.710088 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:24.710067    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" containerName="istio-proxy"
Jul 12 00:03:24.710196 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:24.710100    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" containerName="istio-proxy"
Jul 12 00:03:24.710196 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:24.710191    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bbb7ea2-f913-4cb6-b3ec-eda2b09ed3a0" containerName="istio-proxy"
Jul 12 00:03:24.713382 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:24.713360    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"
Jul 12 00:03:24.715971 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:24.715948    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-14eebdc1-serving-cert\""
Jul 12 00:03:24.726906 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:24.726878    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"]
Jul 12 00:03:24.729238 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:24.729210    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdvlw\" (UniqueName: \"kubernetes.io/projected/3b020a67-7cb9-454c-b06a-f76f5db2ebd5-kube-api-access-rdvlw\") pod \"maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs\" (UID: \"3b020a67-7cb9-454c-b06a-f76f5db2ebd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"
Jul 12 00:03:24.729360 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:24.729278    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b020a67-7cb9-454c-b06a-f76f5db2ebd5-maas-api-tls\") pod \"maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs\" (UID: \"3b020a67-7cb9-454c-b06a-f76f5db2ebd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"
Jul 12 00:03:24.729360 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:24.729316    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b020a67-7cb9-454c-b06a-f76f5db2ebd5-openshift-service-ca\") pod \"maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs\" (UID: \"3b020a67-7cb9-454c-b06a-f76f5db2ebd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"
Jul 12 00:03:24.829974 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:24.829936    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdvlw\" (UniqueName: \"kubernetes.io/projected/3b020a67-7cb9-454c-b06a-f76f5db2ebd5-kube-api-access-rdvlw\") pod \"maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs\" (UID: \"3b020a67-7cb9-454c-b06a-f76f5db2ebd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"
Jul 12 00:03:24.830173 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:24.830008    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b020a67-7cb9-454c-b06a-f76f5db2ebd5-maas-api-tls\") pod \"maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs\" (UID: \"3b020a67-7cb9-454c-b06a-f76f5db2ebd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"
Jul 12 00:03:24.830173 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:24.830051    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b020a67-7cb9-454c-b06a-f76f5db2ebd5-openshift-service-ca\") pod \"maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs\" (UID: \"3b020a67-7cb9-454c-b06a-f76f5db2ebd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"
Jul 12 00:03:24.830806 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:24.830782    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b020a67-7cb9-454c-b06a-f76f5db2ebd5-openshift-service-ca\") pod \"maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs\" (UID: \"3b020a67-7cb9-454c-b06a-f76f5db2ebd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"
Jul 12 00:03:24.832377 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:24.832360    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b020a67-7cb9-454c-b06a-f76f5db2ebd5-maas-api-tls\") pod \"maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs\" (UID: \"3b020a67-7cb9-454c-b06a-f76f5db2ebd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"
Jul 12 00:03:24.840203 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:24.840178    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdvlw\" (UniqueName: \"kubernetes.io/projected/3b020a67-7cb9-454c-b06a-f76f5db2ebd5-kube-api-access-rdvlw\") pod \"maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs\" (UID: \"3b020a67-7cb9-454c-b06a-f76f5db2ebd5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"
Jul 12 00:03:25.025359 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:25.025266    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"
Jul 12 00:03:25.166200 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:25.166165    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"]
Jul 12 00:03:25.169235 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:03:25.169203    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b020a67_7cb9_454c_b06a_f76f5db2ebd5.slice/crio-00f50fad7f3506c02cb7dcc066c215f3cb4c92c2475e495a34184d26c7f3b61e WatchSource:0}: Error finding container 00f50fad7f3506c02cb7dcc066c215f3cb4c92c2475e495a34184d26c7f3b61e: Status 404 returned error can't find the container with id 00f50fad7f3506c02cb7dcc066c215f3cb4c92c2475e495a34184d26c7f3b61e
Jul 12 00:03:25.785047 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:25.784996    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs" event={"ID":"3b020a67-7cb9-454c-b06a-f76f5db2ebd5","Type":"ContainerStarted","Data":"5f0b8e13ab107881ca2f115263306e5bd77f46a20daf3abdb5d8c44130204744"}
Jul 12 00:03:25.785487 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:25.785067    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs" event={"ID":"3b020a67-7cb9-454c-b06a-f76f5db2ebd5","Type":"ContainerStarted","Data":"00f50fad7f3506c02cb7dcc066c215f3cb4c92c2475e495a34184d26c7f3b61e"}
Jul 12 00:03:25.785487 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:25.785174    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"
Jul 12 00:03:25.835547 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:25.835498    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs" podStartSLOduration=1.397425675 podStartE2EDuration="1.8354818s" totalImagesPullingTime="438.056125ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:03:25.834342748 +0000 UTC m=+1935.412435537" watchObservedRunningTime="2026-07-12 00:03:25.8354818 +0000 UTC m=+1935.413574576"
Jul 12 00:03:31.795247 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:31.795214    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"
Jul 12 00:03:38.773007 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:38.772973    2569 patch_prober.go:28] interesting pod/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 12 00:03:38.773347 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:38.773046    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7" podUID="71c29536-da15-43ce-947a-fa8f37d48cc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 12 00:03:38.854475 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:38.854441    2569 generic.go:356] "Generic (PLEG): container finished" podID="71c29536-da15-43ce-947a-fa8f37d48cc5" containerID="4d8f54e77664592b5453a30c870509af4d04255c69500ca64b7bd614ce20b99e" exitCode=0
Jul 12 00:03:38.854644 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:38.854512    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7" event={"ID":"71c29536-da15-43ce-947a-fa8f37d48cc5","Type":"ContainerDied","Data":"4d8f54e77664592b5453a30c870509af4d04255c69500ca64b7bd614ce20b99e"}
Jul 12 00:03:38.854926 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:38.854912    2569 scope.go:117] "RemoveContainer" containerID="4d8f54e77664592b5453a30c870509af4d04255c69500ca64b7bd614ce20b99e"
Jul 12 00:03:38.855914 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:38.855887    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:03:38.855965 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:38.855953    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:03:38.856000 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:38.855986    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:03:39.861607 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:39.861570    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7" event={"ID":"71c29536-da15-43ce-947a-fa8f37d48cc5","Type":"ContainerStarted","Data":"4c00998e7b99c63150cfb3c17ed1bee239396a421307e4d0fa82e20ec9a4c9bb"}
Jul 12 00:03:40.550323 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:40.550282    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:40.550651 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:40.550627    2569 patch_prober.go:28] interesting pod/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 12 00:03:40.550702 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:40.550679    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7" podUID="71c29536-da15-43ce-947a-fa8f37d48cc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 12 00:03:41.550140 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:41.550107    2569 patch_prober.go:28] interesting pod/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 12 00:03:41.550550 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:41.550175    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7" podUID="71c29536-da15-43ce-947a-fa8f37d48cc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 12 00:03:42.550059 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:42.550020    2569 patch_prober.go:28] interesting pod/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 12 00:03:42.550238 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:42.550077    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7" podUID="71c29536-da15-43ce-947a-fa8f37d48cc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 12 00:03:43.549943 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:43.549908    2569 patch_prober.go:28] interesting pod/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 12 00:03:43.550123 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:43.549964    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7" podUID="71c29536-da15-43ce-947a-fa8f37d48cc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 12 00:03:44.553786 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:44.553754    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:44.554195 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:44.554031    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:03:44.554775 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:03:44.554758    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:04:11.157668 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:11.157584    2569 scope.go:117] "RemoveContainer" containerID="aa0c57ffe3edab62745690851411b7b105d1415b66bc4bb23eebe5cf934ad274"
Jul 12 00:04:17.524767 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:17.524727    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14eebdc1-ldwdx"]
Jul 12 00:04:17.528127 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:17.528102    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14eebdc1-ldwdx"
Jul 12 00:04:17.530675 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:17.530655    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\""
Jul 12 00:04:17.536491 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:17.536466    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14eebdc1-ldwdx"]
Jul 12 00:04:17.601607 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:17.601570    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f892bc36-40e9-4162-a5f6-0cc306ac17c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-14eebdc1-ldwdx\" (UID: \"f892bc36-40e9-4162-a5f6-0cc306ac17c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14eebdc1-ldwdx"
Jul 12 00:04:17.702390 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:17.702347    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f892bc36-40e9-4162-a5f6-0cc306ac17c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-14eebdc1-ldwdx\" (UID: \"f892bc36-40e9-4162-a5f6-0cc306ac17c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14eebdc1-ldwdx"
Jul 12 00:04:17.703011 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:17.702988    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f892bc36-40e9-4162-a5f6-0cc306ac17c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-14eebdc1-ldwdx\" (UID: \"f892bc36-40e9-4162-a5f6-0cc306ac17c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14eebdc1-ldwdx"
Jul 12 00:04:17.839178 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:17.839145    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14eebdc1-ldwdx"
Jul 12 00:04:18.177588 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:18.177564    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14eebdc1-ldwdx"]
Jul 12 00:04:18.179411 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:04:18.179389    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf892bc36_40e9_4162_a5f6_0cc306ac17c9.slice/crio-19405b1070a924dc6f1dbec3899d05cba9ca07cbb73e72dd2059332a02a10c7a WatchSource:0}: Error finding container 19405b1070a924dc6f1dbec3899d05cba9ca07cbb73e72dd2059332a02a10c7a: Status 404 returned error can't find the container with id 19405b1070a924dc6f1dbec3899d05cba9ca07cbb73e72dd2059332a02a10c7a
Jul 12 00:04:19.069717 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:19.069681    2569 generic.go:356] "Generic (PLEG): container finished" podID="f892bc36-40e9-4162-a5f6-0cc306ac17c9" containerID="25e7fee3e497b7cb175932b1adeae2ded680922d8460b105357461833ba06dd0" exitCode=0
Jul 12 00:04:19.070158 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:19.069776    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14eebdc1-ldwdx" event={"ID":"f892bc36-40e9-4162-a5f6-0cc306ac17c9","Type":"ContainerDied","Data":"25e7fee3e497b7cb175932b1adeae2ded680922d8460b105357461833ba06dd0"}
Jul 12 00:04:19.070158 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:19.069830    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14eebdc1-ldwdx" event={"ID":"f892bc36-40e9-4162-a5f6-0cc306ac17c9","Type":"ContainerStarted","Data":"19405b1070a924dc6f1dbec3899d05cba9ca07cbb73e72dd2059332a02a10c7a"}
Jul 12 00:04:20.215030 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:20.215006    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14eebdc1-ldwdx"
Jul 12 00:04:20.224405 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:20.224381    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f892bc36-40e9-4162-a5f6-0cc306ac17c9-maas-api-service-ca\") pod \"f892bc36-40e9-4162-a5f6-0cc306ac17c9\" (UID: \"f892bc36-40e9-4162-a5f6-0cc306ac17c9\") "
Jul 12 00:04:20.224748 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:20.224727    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f892bc36-40e9-4162-a5f6-0cc306ac17c9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f892bc36-40e9-4162-a5f6-0cc306ac17c9" (UID: "f892bc36-40e9-4162-a5f6-0cc306ac17c9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:04:20.326185 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:20.326094    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f892bc36-40e9-4162-a5f6-0cc306ac17c9-maas-api-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:04:21.080294 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:21.080259    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14eebdc1-ldwdx" event={"ID":"f892bc36-40e9-4162-a5f6-0cc306ac17c9","Type":"ContainerDied","Data":"19405b1070a924dc6f1dbec3899d05cba9ca07cbb73e72dd2059332a02a10c7a"}
Jul 12 00:04:21.080519 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:21.080313    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19405b1070a924dc6f1dbec3899d05cba9ca07cbb73e72dd2059332a02a10c7a"
Jul 12 00:04:21.080519 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:21.080271    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-14eebdc1-ldwdx"
Jul 12 00:04:27.992029 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:27.991995    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"]
Jul 12 00:04:27.992491 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:27.992192    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs" podUID="3b020a67-7cb9-454c-b06a-f76f5db2ebd5" containerName="maas-api" containerID="cri-o://5f0b8e13ab107881ca2f115263306e5bd77f46a20daf3abdb5d8c44130204744" gracePeriod=30
Jul 12 00:04:28.116434 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:28.116391    2569 generic.go:356] "Generic (PLEG): container finished" podID="3b020a67-7cb9-454c-b06a-f76f5db2ebd5" containerID="5f0b8e13ab107881ca2f115263306e5bd77f46a20daf3abdb5d8c44130204744" exitCode=0
Jul 12 00:04:28.116574 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:28.116460    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs" event={"ID":"3b020a67-7cb9-454c-b06a-f76f5db2ebd5","Type":"ContainerDied","Data":"5f0b8e13ab107881ca2f115263306e5bd77f46a20daf3abdb5d8c44130204744"}
Jul 12 00:04:28.253152 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:28.253082    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"
Jul 12 00:04:28.298387 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:28.298344    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdvlw\" (UniqueName: \"kubernetes.io/projected/3b020a67-7cb9-454c-b06a-f76f5db2ebd5-kube-api-access-rdvlw\") pod \"3b020a67-7cb9-454c-b06a-f76f5db2ebd5\" (UID: \"3b020a67-7cb9-454c-b06a-f76f5db2ebd5\") "
Jul 12 00:04:28.298558 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:28.298520    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b020a67-7cb9-454c-b06a-f76f5db2ebd5-openshift-service-ca\") pod \"3b020a67-7cb9-454c-b06a-f76f5db2ebd5\" (UID: \"3b020a67-7cb9-454c-b06a-f76f5db2ebd5\") "
Jul 12 00:04:28.298558 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:28.298555    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b020a67-7cb9-454c-b06a-f76f5db2ebd5-maas-api-tls\") pod \"3b020a67-7cb9-454c-b06a-f76f5db2ebd5\" (UID: \"3b020a67-7cb9-454c-b06a-f76f5db2ebd5\") "
Jul 12 00:04:28.298870 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:28.298846    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b020a67-7cb9-454c-b06a-f76f5db2ebd5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3b020a67-7cb9-454c-b06a-f76f5db2ebd5" (UID: "3b020a67-7cb9-454c-b06a-f76f5db2ebd5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:04:28.300549 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:28.300522    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b020a67-7cb9-454c-b06a-f76f5db2ebd5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3b020a67-7cb9-454c-b06a-f76f5db2ebd5" (UID: "3b020a67-7cb9-454c-b06a-f76f5db2ebd5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 00:04:28.300691 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:28.300584    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b020a67-7cb9-454c-b06a-f76f5db2ebd5-kube-api-access-rdvlw" (OuterVolumeSpecName: "kube-api-access-rdvlw") pod "3b020a67-7cb9-454c-b06a-f76f5db2ebd5" (UID: "3b020a67-7cb9-454c-b06a-f76f5db2ebd5"). InnerVolumeSpecName "kube-api-access-rdvlw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:04:28.400235 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:28.400201    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdvlw\" (UniqueName: \"kubernetes.io/projected/3b020a67-7cb9-454c-b06a-f76f5db2ebd5-kube-api-access-rdvlw\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:04:28.400235 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:28.400230    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b020a67-7cb9-454c-b06a-f76f5db2ebd5-openshift-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:04:28.400235 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:28.400242    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b020a67-7cb9-454c-b06a-f76f5db2ebd5-maas-api-tls\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:04:29.122593 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:29.122558    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs" event={"ID":"3b020a67-7cb9-454c-b06a-f76f5db2ebd5","Type":"ContainerDied","Data":"00f50fad7f3506c02cb7dcc066c215f3cb4c92c2475e495a34184d26c7f3b61e"}
Jul 12 00:04:29.123076 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:29.122630    2569 scope.go:117] "RemoveContainer" containerID="5f0b8e13ab107881ca2f115263306e5bd77f46a20daf3abdb5d8c44130204744"
Jul 12 00:04:29.123076 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:29.122658    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"
Jul 12 00:04:29.138879 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:29.138843    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"]
Jul 12 00:04:29.142370 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:29.142343    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-14eebdc1-868457c9c8-t9cfs"]
Jul 12 00:04:30.983068 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:04:30.983035    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b020a67-7cb9-454c-b06a-f76f5db2ebd5" path="/var/lib/kubelet/pods/3b020a67-7cb9-454c-b06a-f76f5db2ebd5/volumes"
Jul 12 00:05:44.790692 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:44.790654    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"]
Jul 12 00:05:44.791170 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:44.790921    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7" podUID="71c29536-da15-43ce-947a-fa8f37d48cc5" containerName="istio-proxy" containerID="cri-o://4c00998e7b99c63150cfb3c17ed1bee239396a421307e4d0fa82e20ec9a4c9bb" gracePeriod=30
Jul 12 00:05:50.054008 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.053913    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:05:50.113499 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.113435    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-data\") pod \"71c29536-da15-43ce-947a-fa8f37d48cc5\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") "
Jul 12 00:05:50.113499 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.113480    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-token\") pod \"71c29536-da15-43ce-947a-fa8f37d48cc5\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") "
Jul 12 00:05:50.113499 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.113500    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-envoy\") pod \"71c29536-da15-43ce-947a-fa8f37d48cc5\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") "
Jul 12 00:05:50.113778 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.113526    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-podinfo\") pod \"71c29536-da15-43ce-947a-fa8f37d48cc5\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") "
Jul 12 00:05:50.113778 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.113563    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-workload-socket\") pod \"71c29536-da15-43ce-947a-fa8f37d48cc5\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") "
Jul 12 00:05:50.113778 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.113658    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-credential-socket\") pod \"71c29536-da15-43ce-947a-fa8f37d48cc5\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") "
Jul 12 00:05:50.113778 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.113720    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-workload-certs\") pod \"71c29536-da15-43ce-947a-fa8f37d48cc5\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") "
Jul 12 00:05:50.113778 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.113771    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ktqd\" (UniqueName: \"kubernetes.io/projected/71c29536-da15-43ce-947a-fa8f37d48cc5-kube-api-access-6ktqd\") pod \"71c29536-da15-43ce-947a-fa8f37d48cc5\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") "
Jul 12 00:05:50.114049 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.113796    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71c29536-da15-43ce-947a-fa8f37d48cc5-istiod-ca-cert\") pod \"71c29536-da15-43ce-947a-fa8f37d48cc5\" (UID: \"71c29536-da15-43ce-947a-fa8f37d48cc5\") "
Jul 12 00:05:50.114049 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.113846    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "71c29536-da15-43ce-947a-fa8f37d48cc5" (UID: "71c29536-da15-43ce-947a-fa8f37d48cc5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:05:50.114049 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.113886    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "71c29536-da15-43ce-947a-fa8f37d48cc5" (UID: "71c29536-da15-43ce-947a-fa8f37d48cc5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:05:50.114211 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.114110    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-workload-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:05:50.114211 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.114131    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-credential-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:05:50.114324 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.114252    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-data" (OuterVolumeSpecName: "istio-data") pod "71c29536-da15-43ce-947a-fa8f37d48cc5" (UID: "71c29536-da15-43ce-947a-fa8f37d48cc5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:05:50.114324 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.114266    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71c29536-da15-43ce-947a-fa8f37d48cc5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "71c29536-da15-43ce-947a-fa8f37d48cc5" (UID: "71c29536-da15-43ce-947a-fa8f37d48cc5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:05:50.114469 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.114443    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "71c29536-da15-43ce-947a-fa8f37d48cc5" (UID: "71c29536-da15-43ce-947a-fa8f37d48cc5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:05:50.115856 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.115832    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "71c29536-da15-43ce-947a-fa8f37d48cc5" (UID: "71c29536-da15-43ce-947a-fa8f37d48cc5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 12 00:05:50.115951 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.115857    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-token" (OuterVolumeSpecName: "istio-token") pod "71c29536-da15-43ce-947a-fa8f37d48cc5" (UID: "71c29536-da15-43ce-947a-fa8f37d48cc5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:05:50.115951 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.115865    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "71c29536-da15-43ce-947a-fa8f37d48cc5" (UID: "71c29536-da15-43ce-947a-fa8f37d48cc5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:05:50.115951 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.115864    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71c29536-da15-43ce-947a-fa8f37d48cc5-kube-api-access-6ktqd" (OuterVolumeSpecName: "kube-api-access-6ktqd") pod "71c29536-da15-43ce-947a-fa8f37d48cc5" (UID: "71c29536-da15-43ce-947a-fa8f37d48cc5"). InnerVolumeSpecName "kube-api-access-6ktqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:05:50.214574 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.214544    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ktqd\" (UniqueName: \"kubernetes.io/projected/71c29536-da15-43ce-947a-fa8f37d48cc5-kube-api-access-6ktqd\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:05:50.214574 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.214570    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71c29536-da15-43ce-947a-fa8f37d48cc5-istiod-ca-cert\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:05:50.214574 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.214579    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-data\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:05:50.214798 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.214587    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-token\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:05:50.214798 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.214596    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-envoy\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:05:50.214798 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.214603    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71c29536-da15-43ce-947a-fa8f37d48cc5-istio-podinfo\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:05:50.214798 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.214610    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71c29536-da15-43ce-947a-fa8f37d48cc5-workload-certs\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:05:50.519638 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.519545    2569 generic.go:356] "Generic (PLEG): container finished" podID="71c29536-da15-43ce-947a-fa8f37d48cc5" containerID="4c00998e7b99c63150cfb3c17ed1bee239396a421307e4d0fa82e20ec9a4c9bb" exitCode=0
Jul 12 00:05:50.519638 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.519590    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"
Jul 12 00:05:50.519878 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.519632    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7" event={"ID":"71c29536-da15-43ce-947a-fa8f37d48cc5","Type":"ContainerDied","Data":"4c00998e7b99c63150cfb3c17ed1bee239396a421307e4d0fa82e20ec9a4c9bb"}
Jul 12 00:05:50.519878 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.519696    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7" event={"ID":"71c29536-da15-43ce-947a-fa8f37d48cc5","Type":"ContainerDied","Data":"0a66db8631d31ca47ae1f5f7c329ac6315654d93512f6324bb1d0974826925b6"}
Jul 12 00:05:50.519878 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.519727    2569 scope.go:117] "RemoveContainer" containerID="4c00998e7b99c63150cfb3c17ed1bee239396a421307e4d0fa82e20ec9a4c9bb"
Jul 12 00:05:50.533616 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.533591    2569 scope.go:117] "RemoveContainer" containerID="4d8f54e77664592b5453a30c870509af4d04255c69500ca64b7bd614ce20b99e"
Jul 12 00:05:50.542354 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.542337    2569 scope.go:117] "RemoveContainer" containerID="4c00998e7b99c63150cfb3c17ed1bee239396a421307e4d0fa82e20ec9a4c9bb"
Jul 12 00:05:50.542656 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:05:50.542637    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c00998e7b99c63150cfb3c17ed1bee239396a421307e4d0fa82e20ec9a4c9bb\": container with ID starting with 4c00998e7b99c63150cfb3c17ed1bee239396a421307e4d0fa82e20ec9a4c9bb not found: ID does not exist" containerID="4c00998e7b99c63150cfb3c17ed1bee239396a421307e4d0fa82e20ec9a4c9bb"
Jul 12 00:05:50.542712 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.542664    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c00998e7b99c63150cfb3c17ed1bee239396a421307e4d0fa82e20ec9a4c9bb"} err="failed to get container status \"4c00998e7b99c63150cfb3c17ed1bee239396a421307e4d0fa82e20ec9a4c9bb\": rpc error: code = NotFound desc = could not find container \"4c00998e7b99c63150cfb3c17ed1bee239396a421307e4d0fa82e20ec9a4c9bb\": container with ID starting with 4c00998e7b99c63150cfb3c17ed1bee239396a421307e4d0fa82e20ec9a4c9bb not found: ID does not exist"
Jul 12 00:05:50.542712 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.542680    2569 scope.go:117] "RemoveContainer" containerID="4d8f54e77664592b5453a30c870509af4d04255c69500ca64b7bd614ce20b99e"
Jul 12 00:05:50.542921 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:05:50.542906    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d8f54e77664592b5453a30c870509af4d04255c69500ca64b7bd614ce20b99e\": container with ID starting with 4d8f54e77664592b5453a30c870509af4d04255c69500ca64b7bd614ce20b99e not found: ID does not exist" containerID="4d8f54e77664592b5453a30c870509af4d04255c69500ca64b7bd614ce20b99e"
Jul 12 00:05:50.542967 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.542926    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d8f54e77664592b5453a30c870509af4d04255c69500ca64b7bd614ce20b99e"} err="failed to get container status \"4d8f54e77664592b5453a30c870509af4d04255c69500ca64b7bd614ce20b99e\": rpc error: code = NotFound desc = could not find container \"4d8f54e77664592b5453a30c870509af4d04255c69500ca64b7bd614ce20b99e\": container with ID starting with 4d8f54e77664592b5453a30c870509af4d04255c69500ca64b7bd614ce20b99e not found: ID does not exist"
Jul 12 00:05:50.543618 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.543595    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"]
Jul 12 00:05:50.546539 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.546518    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-14eebdc1-openshift-default-698c4fd74c-6j2k7"]
Jul 12 00:05:50.983113 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:05:50.983079    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71c29536-da15-43ce-947a-fa8f37d48cc5" path="/var/lib/kubelet/pods/71c29536-da15-43ce-947a-fa8f37d48cc5/volumes"
Jul 12 00:06:11.188665 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:11.188628    2569 scope.go:117] "RemoveContainer" containerID="e787a0b80f158cb3a8da550d1124d2fa349b3968752eb8d93068a0faf1890fee"
Jul 12 00:06:11.199595 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:11.199567    2569 scope.go:117] "RemoveContainer" containerID="71b3b8f593e3b08bd7154793220f9286921def277dddfa83277c9315cb9b6048"
Jul 12 00:06:11.919408 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:11.919380    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 12 00:06:11.922990 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:11.922967    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 12 00:06:38.780038 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.780004    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"]
Jul 12 00:06:38.780532 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.780516    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71c29536-da15-43ce-947a-fa8f37d48cc5" containerName="istio-proxy"
Jul 12 00:06:38.780532 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.780533    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="71c29536-da15-43ce-947a-fa8f37d48cc5" containerName="istio-proxy"
Jul 12 00:06:38.780634 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.780541    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b020a67-7cb9-454c-b06a-f76f5db2ebd5" containerName="maas-api"
Jul 12 00:06:38.780634 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.780548    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b020a67-7cb9-454c-b06a-f76f5db2ebd5" containerName="maas-api"
Jul 12 00:06:38.780634 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.780554    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f892bc36-40e9-4162-a5f6-0cc306ac17c9" containerName="revoke-keys"
Jul 12 00:06:38.780634 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.780559    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f892bc36-40e9-4162-a5f6-0cc306ac17c9" containerName="revoke-keys"
Jul 12 00:06:38.780634 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.780566    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71c29536-da15-43ce-947a-fa8f37d48cc5" containerName="istio-proxy"
Jul 12 00:06:38.780634 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.780571    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="71c29536-da15-43ce-947a-fa8f37d48cc5" containerName="istio-proxy"
Jul 12 00:06:38.780634 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.780636    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="71c29536-da15-43ce-947a-fa8f37d48cc5" containerName="istio-proxy"
Jul 12 00:06:38.780852 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.780644    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b020a67-7cb9-454c-b06a-f76f5db2ebd5" containerName="maas-api"
Jul 12 00:06:38.780852 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.780650    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="71c29536-da15-43ce-947a-fa8f37d48cc5" containerName="istio-proxy"
Jul 12 00:06:38.780852 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.780658    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f892bc36-40e9-4162-a5f6-0cc306ac17c9" containerName="revoke-keys"
Jul 12 00:06:38.783673 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.783656    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:38.786199 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.786176    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-7007a3-openshift-default-dockercfg-bwmxc\""
Jul 12 00:06:38.795787 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.795767    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"]
Jul 12 00:06:38.933664 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.933621    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-workload-certs\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:38.933664 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.933664    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-podinfo\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:38.933946 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.933684    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea6024f6-0d9c-4528-b1d5-895623a932d1-istiod-ca-cert\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:38.933946 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.933738    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbhwd\" (UniqueName: \"kubernetes.io/projected/ea6024f6-0d9c-4528-b1d5-895623a932d1-kube-api-access-sbhwd\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:38.933946 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.933806    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-workload-socket\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:38.933946 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.933833    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-token\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:38.933946 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.933860    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-envoy\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:38.933946 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.933915    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-credential-socket\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:38.934186 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:38.933976    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-data\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.035288 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.035195    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-workload-socket\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.035288 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.035232    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-token\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.035288 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.035262    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-envoy\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.035627 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.035314    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-credential-socket\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.035627 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.035366    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-data\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.035627 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.035440    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-workload-certs\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.035627 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.035486    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-podinfo\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.035627 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.035518    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea6024f6-0d9c-4528-b1d5-895623a932d1-istiod-ca-cert\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.035627 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.035558    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbhwd\" (UniqueName: \"kubernetes.io/projected/ea6024f6-0d9c-4528-b1d5-895623a932d1-kube-api-access-sbhwd\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.035928 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.035667    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-workload-socket\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.035928 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.035725    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-credential-socket\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.035928 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.035812    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-workload-certs\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.036048 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.035988    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-data\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.036160 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.036141    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea6024f6-0d9c-4528-b1d5-895623a932d1-istiod-ca-cert\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.037506 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.037487    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-envoy\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.037851 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.037832    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-podinfo\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.042858 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.042831    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-token\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.043609 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.043589    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbhwd\" (UniqueName: \"kubernetes.io/projected/ea6024f6-0d9c-4528-b1d5-895623a932d1-kube-api-access-sbhwd\") pod \"e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") " pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.094542 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.094495    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:39.228892 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.228863    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"]
Jul 12 00:06:39.230938 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:06:39.230907    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea6024f6_0d9c_4528_b1d5_895623a932d1.slice/crio-de0fbeb953fc2bb91ef76312c3a45f935e0b06ecca6e9ebf94cf2cb5258b43fa WatchSource:0}: Error finding container de0fbeb953fc2bb91ef76312c3a45f935e0b06ecca6e9ebf94cf2cb5258b43fa: Status 404 returned error can't find the container with id de0fbeb953fc2bb91ef76312c3a45f935e0b06ecca6e9ebf94cf2cb5258b43fa
Jul 12 00:06:39.233542 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.233508    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:06:39.233641 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.233596    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:06:39.233641 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.233625    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:06:39.755376 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.755340    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42" event={"ID":"ea6024f6-0d9c-4528-b1d5-895623a932d1","Type":"ContainerStarted","Data":"7d8d069a2c38b1d015efe4f60b85a9f97230f14b4fe66fedf859d13f8aa64b37"}
Jul 12 00:06:39.755595 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.755394    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42" event={"ID":"ea6024f6-0d9c-4528-b1d5-895623a932d1","Type":"ContainerStarted","Data":"de0fbeb953fc2bb91ef76312c3a45f935e0b06ecca6e9ebf94cf2cb5258b43fa"}
Jul 12 00:06:39.774858 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:39.774804    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42" podStartSLOduration=1.77478931 podStartE2EDuration="1.77478931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:06:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:06:39.772675232 +0000 UTC m=+2129.350768008" watchObservedRunningTime="2026-07-12 00:06:39.77478931 +0000 UTC m=+2129.352882086"
Jul 12 00:06:40.094985 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:40.094947    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:40.099666 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:40.099640    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:40.761896 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:40.761864    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:40.762776 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:40.762756    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:41.788276 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:41.788243    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"]
Jul 12 00:06:41.791760 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:41.791742    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"
Jul 12 00:06:41.794217 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:41.794191    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-7007a3-serving-cert\""
Jul 12 00:06:41.801308 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:41.801283    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"]
Jul 12 00:06:41.860251 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:41.860216    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bae18e8b-0719-4aac-83da-18d9945f9b64-maas-api-tls\") pod \"maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw\" (UID: \"bae18e8b-0719-4aac-83da-18d9945f9b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"
Jul 12 00:06:41.860442 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:41.860281    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bae18e8b-0719-4aac-83da-18d9945f9b64-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw\" (UID: \"bae18e8b-0719-4aac-83da-18d9945f9b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"
Jul 12 00:06:41.860502 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:41.860454    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs4d9\" (UniqueName: \"kubernetes.io/projected/bae18e8b-0719-4aac-83da-18d9945f9b64-kube-api-access-gs4d9\") pod \"maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw\" (UID: \"bae18e8b-0719-4aac-83da-18d9945f9b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"
Jul 12 00:06:41.961128 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:41.961099    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bae18e8b-0719-4aac-83da-18d9945f9b64-maas-api-tls\") pod \"maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw\" (UID: \"bae18e8b-0719-4aac-83da-18d9945f9b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"
Jul 12 00:06:41.961332 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:41.961141    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bae18e8b-0719-4aac-83da-18d9945f9b64-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw\" (UID: \"bae18e8b-0719-4aac-83da-18d9945f9b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"
Jul 12 00:06:41.961332 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:41.961163    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs4d9\" (UniqueName: \"kubernetes.io/projected/bae18e8b-0719-4aac-83da-18d9945f9b64-kube-api-access-gs4d9\") pod \"maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw\" (UID: \"bae18e8b-0719-4aac-83da-18d9945f9b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"
Jul 12 00:06:41.961856 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:41.961832    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bae18e8b-0719-4aac-83da-18d9945f9b64-openshift-service-ca\") pod \"maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw\" (UID: \"bae18e8b-0719-4aac-83da-18d9945f9b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"
Jul 12 00:06:41.963575 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:41.963555    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bae18e8b-0719-4aac-83da-18d9945f9b64-maas-api-tls\") pod \"maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw\" (UID: \"bae18e8b-0719-4aac-83da-18d9945f9b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"
Jul 12 00:06:41.976966 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:41.976944    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs4d9\" (UniqueName: \"kubernetes.io/projected/bae18e8b-0719-4aac-83da-18d9945f9b64-kube-api-access-gs4d9\") pod \"maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw\" (UID: \"bae18e8b-0719-4aac-83da-18d9945f9b64\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"
Jul 12 00:06:42.106446 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:42.106401    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"
Jul 12 00:06:42.238319 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:42.238294    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"]
Jul 12 00:06:42.240018 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:06:42.239989    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbae18e8b_0719_4aac_83da_18d9945f9b64.slice/crio-b3ab04410ea713ba0c6c2203f855b96002c682de0af2b99bc370e3051c50d665 WatchSource:0}: Error finding container b3ab04410ea713ba0c6c2203f855b96002c682de0af2b99bc370e3051c50d665: Status 404 returned error can't find the container with id b3ab04410ea713ba0c6c2203f855b96002c682de0af2b99bc370e3051c50d665
Jul 12 00:06:42.773296 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:42.773256    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw" event={"ID":"bae18e8b-0719-4aac-83da-18d9945f9b64","Type":"ContainerStarted","Data":"0eb27c60cf0df07c850c4f8cc41e19ca8ef58e273a36ed88678470857592a99d"}
Jul 12 00:06:42.773505 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:42.773316    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw" event={"ID":"bae18e8b-0719-4aac-83da-18d9945f9b64","Type":"ContainerStarted","Data":"b3ab04410ea713ba0c6c2203f855b96002c682de0af2b99bc370e3051c50d665"}
Jul 12 00:06:42.773591 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:42.773530    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"
Jul 12 00:06:42.790498 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:42.790453    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw" podStartSLOduration=1.386833886 podStartE2EDuration="1.790439106s" totalImagesPullingTime="403.60522ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:06:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:06:42.788540612 +0000 UTC m=+2132.366633385" watchObservedRunningTime="2026-07-12 00:06:42.790439106 +0000 UTC m=+2132.368531873"
Jul 12 00:06:46.802259 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:46.802226    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"]
Jul 12 00:06:46.805714 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:46.805691    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:46.811456 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:46.808546    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ad0409-openshift-default-dockercfg-xccgt\""
Jul 12 00:06:46.821907 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:46.821874    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"]
Jul 12 00:06:46.910418 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:46.910375    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-workload-socket\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:46.910644 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:46.910538    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-credential-socket\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:46.910644 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:46.910597    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-workload-certs\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:46.910794 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:46.910667    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkjmz\" (UniqueName: \"kubernetes.io/projected/f297935c-26a7-4c5e-8e98-41e45e73efad-kube-api-access-zkjmz\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:46.910794 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:46.910710    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-token\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:46.910794 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:46.910744    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-envoy\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:46.910794 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:46.910767    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f297935c-26a7-4c5e-8e98-41e45e73efad-istiod-ca-cert\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:46.911009 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:46.910807    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-data\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:46.911009 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:46.910841    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-podinfo\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.011637 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.011607    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-workload-certs\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.011831 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.011669    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkjmz\" (UniqueName: \"kubernetes.io/projected/f297935c-26a7-4c5e-8e98-41e45e73efad-kube-api-access-zkjmz\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.011831 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.011697    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-token\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.011831 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.011800    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-envoy\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.012004 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.011849    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f297935c-26a7-4c5e-8e98-41e45e73efad-istiod-ca-cert\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.012004 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.011892    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-data\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.012004 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.011936    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-podinfo\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.012004 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.011973    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-workload-socket\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.012221 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.012033    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-credential-socket\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.012221 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.012064    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-workload-certs\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.012364 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.012335    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-data\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.012455 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.012372    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-credential-socket\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.012455 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.012415    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-workload-socket\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.012611 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.012591    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f297935c-26a7-4c5e-8e98-41e45e73efad-istiod-ca-cert\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.014047 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.014030    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-envoy\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.014333 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.014314    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-podinfo\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.019555 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.019532    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-token\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.020021 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.020000    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkjmz\" (UniqueName: \"kubernetes.io/projected/f297935c-26a7-4c5e-8e98-41e45e73efad-kube-api-access-zkjmz\") pod \"e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") " pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.124525 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.124413    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:47.259525 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.259487    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"]
Jul 12 00:06:47.261111 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:06:47.261085    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf297935c_26a7_4c5e_8e98_41e45e73efad.slice/crio-dacb3526946049980004dd6143663c37cebf9906c61a2f9449c3bbd230813e0c WatchSource:0}: Error finding container dacb3526946049980004dd6143663c37cebf9906c61a2f9449c3bbd230813e0c: Status 404 returned error can't find the container with id dacb3526946049980004dd6143663c37cebf9906c61a2f9449c3bbd230813e0c
Jul 12 00:06:47.263446 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.263398    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:06:47.263529 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.263479    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:06:47.263529 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.263507    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:06:47.800540 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.800496    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb" event={"ID":"f297935c-26a7-4c5e-8e98-41e45e73efad","Type":"ContainerStarted","Data":"c613d8e02224199a7c46b0bd378e2f548d77971219bfbbd24c9ca424e0788c88"}
Jul 12 00:06:47.800744 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.800556    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb" event={"ID":"f297935c-26a7-4c5e-8e98-41e45e73efad","Type":"ContainerStarted","Data":"dacb3526946049980004dd6143663c37cebf9906c61a2f9449c3bbd230813e0c"}
Jul 12 00:06:47.822597 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:47.822546    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb" podStartSLOduration=1.822533169 podStartE2EDuration="1.822533169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:06:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:06:47.821981521 +0000 UTC m=+2137.400074296" watchObservedRunningTime="2026-07-12 00:06:47.822533169 +0000 UTC m=+2137.400625944"
Jul 12 00:06:48.125020 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:48.124930    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:48.129979 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:48.129947    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:48.785437 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:48.785396    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"
Jul 12 00:06:48.806034 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:48.806006    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:48.807004 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:48.806986    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:06:56.802642 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:56.802589    2569 patch_prober.go:28] interesting pod/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 12 00:06:56.803049 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:56.802663    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42" podUID="ea6024f6-0d9c-4528-b1d5-895623a932d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 12 00:06:57.859752 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:57.859704    2569 generic.go:356] "Generic (PLEG): container finished" podID="ea6024f6-0d9c-4528-b1d5-895623a932d1" containerID="7d8d069a2c38b1d015efe4f60b85a9f97230f14b4fe66fedf859d13f8aa64b37" exitCode=0
Jul 12 00:06:57.860260 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:57.859816    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42" event={"ID":"ea6024f6-0d9c-4528-b1d5-895623a932d1","Type":"ContainerDied","Data":"7d8d069a2c38b1d015efe4f60b85a9f97230f14b4fe66fedf859d13f8aa64b37"}
Jul 12 00:06:57.873479 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:57.866735    2569 scope.go:117] "RemoveContainer" containerID="7d8d069a2c38b1d015efe4f60b85a9f97230f14b4fe66fedf859d13f8aa64b37"
Jul 12 00:06:57.878142 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:57.878056    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:06:57.884630 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:57.884319    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:06:57.884630 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:57.884403    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:06:58.870414 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:58.870377    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42" event={"ID":"ea6024f6-0d9c-4528-b1d5-895623a932d1","Type":"ContainerStarted","Data":"3aff1f12307a7612a70e72faaffc84721f54186f921a64a9565ac3dbfef2d48c"}
Jul 12 00:06:59.095626 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:59.095533    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:06:59.095626 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:59.095573    2569 patch_prober.go:28] interesting pod/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Jul 12 00:06:59.095626 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:06:59.095619    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42" podUID="ea6024f6-0d9c-4528-b1d5-895623a932d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Jul 12 00:07:00.095361 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:00.095317    2569 patch_prober.go:28] interesting pod/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Jul 12 00:07:00.096074 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:00.096038    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42" podUID="ea6024f6-0d9c-4528-b1d5-895623a932d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Jul 12 00:07:01.095894 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:01.095847    2569 patch_prober.go:28] interesting pod/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Jul 12 00:07:01.096257 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:01.095931    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42" podUID="ea6024f6-0d9c-4528-b1d5-895623a932d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Jul 12 00:07:02.095644 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:02.095610    2569 patch_prober.go:28] interesting pod/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Jul 12 00:07:02.095888 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:02.095863    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42" podUID="ea6024f6-0d9c-4528-b1d5-895623a932d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Jul 12 00:07:03.099825 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:03.099786    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:07:03.100316 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:03.100085    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:07:03.100761 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:03.100746    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:07:03.953306 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:03.952665    2569 patch_prober.go:28] interesting pod/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": read tcp 10.133.0.2:42160->10.133.0.60:15021: read: connection reset by peer" start-of-body=
Jul 12 00:07:03.953306 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:03.952770    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb" podUID="f297935c-26a7-4c5e-8e98-41e45e73efad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": read tcp 10.133.0.2:42160->10.133.0.60:15021: read: connection reset by peer"
Jul 12 00:07:04.911240 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:04.911209    2569 generic.go:356] "Generic (PLEG): container finished" podID="f297935c-26a7-4c5e-8e98-41e45e73efad" containerID="c613d8e02224199a7c46b0bd378e2f548d77971219bfbbd24c9ca424e0788c88" exitCode=0
Jul 12 00:07:04.911762 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:04.911292    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb" event={"ID":"f297935c-26a7-4c5e-8e98-41e45e73efad","Type":"ContainerDied","Data":"c613d8e02224199a7c46b0bd378e2f548d77971219bfbbd24c9ca424e0788c88"}
Jul 12 00:07:04.911762 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:04.911757    2569 scope.go:117] "RemoveContainer" containerID="c613d8e02224199a7c46b0bd378e2f548d77971219bfbbd24c9ca424e0788c88"
Jul 12 00:07:04.912774 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:04.912742    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:07:04.912844 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:04.912814    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:07:04.912882 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:04.912854    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:07:05.918032 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:05.917994    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb" event={"ID":"f297935c-26a7-4c5e-8e98-41e45e73efad","Type":"ContainerStarted","Data":"6781f42961d1241ba22239467380b8483997c9014f7ef1bd2f3f7fa9bf583384"}
Jul 12 00:07:06.124997 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:06.124961    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:07:06.125190 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:06.125163    2569 patch_prober.go:28] interesting pod/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 12 00:07:06.125268 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:06.125222    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb" podUID="f297935c-26a7-4c5e-8e98-41e45e73efad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 12 00:07:07.125390 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:07.125356    2569 patch_prober.go:28] interesting pod/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 12 00:07:07.125795 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:07.125418    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb" podUID="f297935c-26a7-4c5e-8e98-41e45e73efad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 12 00:07:08.125370 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:08.125333    2569 patch_prober.go:28] interesting pod/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 12 00:07:08.125583 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:08.125390    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb" podUID="f297935c-26a7-4c5e-8e98-41e45e73efad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 12 00:07:09.125781 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:09.125738    2569 patch_prober.go:28] interesting pod/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 12 00:07:09.126165 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:09.125798    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb" podUID="f297935c-26a7-4c5e-8e98-41e45e73efad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 12 00:07:10.128137 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:10.128105    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:07:10.128534 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:10.128363    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:07:10.129036 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:07:10.129019    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:08:09.438225 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:09.438189    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff"]
Jul 12 00:08:09.442533 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:09.442511    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff"
Jul 12 00:08:09.445351 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:09.445316    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\""
Jul 12 00:08:09.448189 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:09.448168    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff"]
Jul 12 00:08:09.586115 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:09.586073    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3649950-0cd2-4a55-8582-7435fb3bc456-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff\" (UID: \"e3649950-0cd2-4a55-8582-7435fb3bc456\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff"
Jul 12 00:08:09.687036 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:09.686997    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3649950-0cd2-4a55-8582-7435fb3bc456-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff\" (UID: \"e3649950-0cd2-4a55-8582-7435fb3bc456\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff"
Jul 12 00:08:09.687672 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:09.687652    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3649950-0cd2-4a55-8582-7435fb3bc456-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff\" (UID: \"e3649950-0cd2-4a55-8582-7435fb3bc456\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff"
Jul 12 00:08:09.755293 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:09.755195    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff"
Jul 12 00:08:09.890382 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:09.890343    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff"]
Jul 12 00:08:09.892703 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:08:09.892676    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3649950_0cd2_4a55_8582_7435fb3bc456.slice/crio-b0c791f2b64af2d86d4e04cd760aad5c9f646493132473edc2b1053336894148 WatchSource:0}: Error finding container b0c791f2b64af2d86d4e04cd760aad5c9f646493132473edc2b1053336894148: Status 404 returned error can't find the container with id b0c791f2b64af2d86d4e04cd760aad5c9f646493132473edc2b1053336894148
Jul 12 00:08:10.233577 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:10.233532    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff" event={"ID":"e3649950-0cd2-4a55-8582-7435fb3bc456","Type":"ContainerStarted","Data":"cda7a1ad34b9f095f81ee45c8df90f1ba840bf0012db880d5eea21b59ef256d8"}
Jul 12 00:08:10.233788 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:10.233601    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff" event={"ID":"e3649950-0cd2-4a55-8582-7435fb3bc456","Type":"ContainerStarted","Data":"b0c791f2b64af2d86d4e04cd760aad5c9f646493132473edc2b1053336894148"}
Jul 12 00:08:10.248543 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:10.248494    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff" podStartSLOduration=1.248479718 podStartE2EDuration="1.248479718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:08:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:08:10.247078332 +0000 UTC m=+2219.825171119" watchObservedRunningTime="2026-07-12 00:08:10.248479718 +0000 UTC m=+2219.826572494"
Jul 12 00:08:11.239462 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:11.239412    2569 generic.go:356] "Generic (PLEG): container finished" podID="e3649950-0cd2-4a55-8582-7435fb3bc456" containerID="cda7a1ad34b9f095f81ee45c8df90f1ba840bf0012db880d5eea21b59ef256d8" exitCode=0
Jul 12 00:08:11.239847 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:11.239497    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff" event={"ID":"e3649950-0cd2-4a55-8582-7435fb3bc456","Type":"ContainerDied","Data":"cda7a1ad34b9f095f81ee45c8df90f1ba840bf0012db880d5eea21b59ef256d8"}
Jul 12 00:08:12.379319 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:12.379292    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff"
Jul 12 00:08:12.511722 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:12.511616    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3649950-0cd2-4a55-8582-7435fb3bc456-maas-api-service-ca\") pod \"e3649950-0cd2-4a55-8582-7435fb3bc456\" (UID: \"e3649950-0cd2-4a55-8582-7435fb3bc456\") "
Jul 12 00:08:12.511989 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:12.511967    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3649950-0cd2-4a55-8582-7435fb3bc456-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e3649950-0cd2-4a55-8582-7435fb3bc456" (UID: "e3649950-0cd2-4a55-8582-7435fb3bc456"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:08:12.612614 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:12.612579    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3649950-0cd2-4a55-8582-7435fb3bc456-maas-api-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:08:13.251407 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:13.251324    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff"
Jul 12 00:08:13.251581 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:13.251321    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff" event={"ID":"e3649950-0cd2-4a55-8582-7435fb3bc456","Type":"ContainerDied","Data":"b0c791f2b64af2d86d4e04cd760aad5c9f646493132473edc2b1053336894148"}
Jul 12 00:08:13.251581 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:13.251469    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0c791f2b64af2d86d4e04cd760aad5c9f646493132473edc2b1053336894148"
Jul 12 00:08:19.953100 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:19.953060    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"]
Jul 12 00:08:19.953542 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:19.953331    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw" podUID="bae18e8b-0719-4aac-83da-18d9945f9b64" containerName="maas-api" containerID="cri-o://0eb27c60cf0df07c850c4f8cc41e19ca8ef58e273a36ed88678470857592a99d" gracePeriod=30
Jul 12 00:08:20.210552 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.210486    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"
Jul 12 00:08:20.277334 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.277301    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bae18e8b-0719-4aac-83da-18d9945f9b64-openshift-service-ca\") pod \"bae18e8b-0719-4aac-83da-18d9945f9b64\" (UID: \"bae18e8b-0719-4aac-83da-18d9945f9b64\") "
Jul 12 00:08:20.277334 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.277346    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gs4d9\" (UniqueName: \"kubernetes.io/projected/bae18e8b-0719-4aac-83da-18d9945f9b64-kube-api-access-gs4d9\") pod \"bae18e8b-0719-4aac-83da-18d9945f9b64\" (UID: \"bae18e8b-0719-4aac-83da-18d9945f9b64\") "
Jul 12 00:08:20.277553 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.277487    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bae18e8b-0719-4aac-83da-18d9945f9b64-maas-api-tls\") pod \"bae18e8b-0719-4aac-83da-18d9945f9b64\" (UID: \"bae18e8b-0719-4aac-83da-18d9945f9b64\") "
Jul 12 00:08:20.277729 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.277696    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bae18e8b-0719-4aac-83da-18d9945f9b64-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bae18e8b-0719-4aac-83da-18d9945f9b64" (UID: "bae18e8b-0719-4aac-83da-18d9945f9b64"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:08:20.279510 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.279484    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bae18e8b-0719-4aac-83da-18d9945f9b64-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bae18e8b-0719-4aac-83da-18d9945f9b64" (UID: "bae18e8b-0719-4aac-83da-18d9945f9b64"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 00:08:20.279592 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.279506    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bae18e8b-0719-4aac-83da-18d9945f9b64-kube-api-access-gs4d9" (OuterVolumeSpecName: "kube-api-access-gs4d9") pod "bae18e8b-0719-4aac-83da-18d9945f9b64" (UID: "bae18e8b-0719-4aac-83da-18d9945f9b64"). InnerVolumeSpecName "kube-api-access-gs4d9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:08:20.289220 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.289196    2569 generic.go:356] "Generic (PLEG): container finished" podID="bae18e8b-0719-4aac-83da-18d9945f9b64" containerID="0eb27c60cf0df07c850c4f8cc41e19ca8ef58e273a36ed88678470857592a99d" exitCode=0
Jul 12 00:08:20.289350 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.289248    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"
Jul 12 00:08:20.289350 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.289279    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw" event={"ID":"bae18e8b-0719-4aac-83da-18d9945f9b64","Type":"ContainerDied","Data":"0eb27c60cf0df07c850c4f8cc41e19ca8ef58e273a36ed88678470857592a99d"}
Jul 12 00:08:20.289350 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.289336    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw" event={"ID":"bae18e8b-0719-4aac-83da-18d9945f9b64","Type":"ContainerDied","Data":"b3ab04410ea713ba0c6c2203f855b96002c682de0af2b99bc370e3051c50d665"}
Jul 12 00:08:20.289592 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.289360    2569 scope.go:117] "RemoveContainer" containerID="0eb27c60cf0df07c850c4f8cc41e19ca8ef58e273a36ed88678470857592a99d"
Jul 12 00:08:20.298146 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.298126    2569 scope.go:117] "RemoveContainer" containerID="0eb27c60cf0df07c850c4f8cc41e19ca8ef58e273a36ed88678470857592a99d"
Jul 12 00:08:20.298392 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:08:20.298373    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eb27c60cf0df07c850c4f8cc41e19ca8ef58e273a36ed88678470857592a99d\": container with ID starting with 0eb27c60cf0df07c850c4f8cc41e19ca8ef58e273a36ed88678470857592a99d not found: ID does not exist" containerID="0eb27c60cf0df07c850c4f8cc41e19ca8ef58e273a36ed88678470857592a99d"
Jul 12 00:08:20.298445 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.298400    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eb27c60cf0df07c850c4f8cc41e19ca8ef58e273a36ed88678470857592a99d"} err="failed to get container status \"0eb27c60cf0df07c850c4f8cc41e19ca8ef58e273a36ed88678470857592a99d\": rpc error: code = NotFound desc = could not find container \"0eb27c60cf0df07c850c4f8cc41e19ca8ef58e273a36ed88678470857592a99d\": container with ID starting with 0eb27c60cf0df07c850c4f8cc41e19ca8ef58e273a36ed88678470857592a99d not found: ID does not exist"
Jul 12 00:08:20.311586 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.311549    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"]
Jul 12 00:08:20.313739 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.313716    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-7007a3-67fbf47c7f-lk6dw"]
Jul 12 00:08:20.378548 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.378512    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bae18e8b-0719-4aac-83da-18d9945f9b64-maas-api-tls\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:08:20.378548 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.378543    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bae18e8b-0719-4aac-83da-18d9945f9b64-openshift-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:08:20.378548 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.378552    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gs4d9\" (UniqueName: \"kubernetes.io/projected/bae18e8b-0719-4aac-83da-18d9945f9b64-kube-api-access-gs4d9\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:08:20.984441 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:20.984396    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bae18e8b-0719-4aac-83da-18d9945f9b64" path="/var/lib/kubelet/pods/bae18e8b-0719-4aac-83da-18d9945f9b64/volumes"
Jul 12 00:08:35.227382 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:35.227350    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"]
Jul 12 00:08:35.227860 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:35.227610    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42" podUID="ea6024f6-0d9c-4528-b1d5-895623a932d1" containerName="istio-proxy" containerID="cri-o://3aff1f12307a7612a70e72faaffc84721f54186f921a64a9565ac3dbfef2d48c" gracePeriod=30
Jul 12 00:08:35.612787 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:35.612739    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ad0409-ntmcm"]
Jul 12 00:08:35.613285 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:35.613269    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bae18e8b-0719-4aac-83da-18d9945f9b64" containerName="maas-api"
Jul 12 00:08:35.613331 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:35.613289    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="bae18e8b-0719-4aac-83da-18d9945f9b64" containerName="maas-api"
Jul 12 00:08:35.613331 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:35.613324    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3649950-0cd2-4a55-8582-7435fb3bc456" containerName="revoke-keys"
Jul 12 00:08:35.613393 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:35.613333    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3649950-0cd2-4a55-8582-7435fb3bc456" containerName="revoke-keys"
Jul 12 00:08:35.613452 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:35.613416    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="bae18e8b-0719-4aac-83da-18d9945f9b64" containerName="maas-api"
Jul 12 00:08:35.613452 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:35.613443    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3649950-0cd2-4a55-8582-7435fb3bc456" containerName="revoke-keys"
Jul 12 00:08:35.616959 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:35.616939    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ad0409-ntmcm"
Jul 12 00:08:35.619617 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:35.619593    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\""
Jul 12 00:08:35.621272 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:35.621245    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ad0409-ntmcm"]
Jul 12 00:08:35.710904 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:35.710868    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1381f12e-957e-4f32-9883-a97016a0f7e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ad0409-ntmcm\" (UID: \"1381f12e-957e-4f32-9883-a97016a0f7e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ad0409-ntmcm"
Jul 12 00:08:35.811941 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:35.811899    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1381f12e-957e-4f32-9883-a97016a0f7e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ad0409-ntmcm\" (UID: \"1381f12e-957e-4f32-9883-a97016a0f7e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ad0409-ntmcm"
Jul 12 00:08:35.812585 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:35.812559    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1381f12e-957e-4f32-9883-a97016a0f7e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ad0409-ntmcm\" (UID: \"1381f12e-957e-4f32-9883-a97016a0f7e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ad0409-ntmcm"
Jul 12 00:08:35.928599 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:35.928514    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ad0409-ntmcm"
Jul 12 00:08:36.277108 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:36.273309    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ad0409-ntmcm"]
Jul 12 00:08:36.278484 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:08:36.278454    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1381f12e_957e_4f32_9883_a97016a0f7e2.slice/crio-31b0c45e0bbe46b7b042958d12658bf8dbd4f775985920a2d3123f005e7c0606 WatchSource:0}: Error finding container 31b0c45e0bbe46b7b042958d12658bf8dbd4f775985920a2d3123f005e7c0606: Status 404 returned error can't find the container with id 31b0c45e0bbe46b7b042958d12658bf8dbd4f775985920a2d3123f005e7c0606
Jul 12 00:08:36.370625 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:36.370586    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ad0409-ntmcm" event={"ID":"1381f12e-957e-4f32-9883-a97016a0f7e2","Type":"ContainerStarted","Data":"31b0c45e0bbe46b7b042958d12658bf8dbd4f775985920a2d3123f005e7c0606"}
Jul 12 00:08:37.376917 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:37.376878    2569 generic.go:356] "Generic (PLEG): container finished" podID="1381f12e-957e-4f32-9883-a97016a0f7e2" containerID="067537e7f636583405e11a54bb7176d0c91662429b86ef1380ccefd319172bce" exitCode=0
Jul 12 00:08:37.377290 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:37.376930    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ad0409-ntmcm" event={"ID":"1381f12e-957e-4f32-9883-a97016a0f7e2","Type":"ContainerDied","Data":"067537e7f636583405e11a54bb7176d0c91662429b86ef1380ccefd319172bce"}
Jul 12 00:08:38.511277 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:38.511253    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ad0409-ntmcm"
Jul 12 00:08:38.639376 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:38.639281    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1381f12e-957e-4f32-9883-a97016a0f7e2-maas-api-service-ca\") pod \"1381f12e-957e-4f32-9883-a97016a0f7e2\" (UID: \"1381f12e-957e-4f32-9883-a97016a0f7e2\") "
Jul 12 00:08:38.639675 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:38.639653    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1381f12e-957e-4f32-9883-a97016a0f7e2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1381f12e-957e-4f32-9883-a97016a0f7e2" (UID: "1381f12e-957e-4f32-9883-a97016a0f7e2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:08:38.740329 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:38.740293    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1381f12e-957e-4f32-9883-a97016a0f7e2-maas-api-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:08:39.387923 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:39.387893    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ad0409-ntmcm"
Jul 12 00:08:39.388121 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:39.387892    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ad0409-ntmcm" event={"ID":"1381f12e-957e-4f32-9883-a97016a0f7e2","Type":"ContainerDied","Data":"31b0c45e0bbe46b7b042958d12658bf8dbd4f775985920a2d3123f005e7c0606"}
Jul 12 00:08:39.388121 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:39.388024    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="31b0c45e0bbe46b7b042958d12658bf8dbd4f775985920a2d3123f005e7c0606"
Jul 12 00:08:40.396349 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.396319    2569 generic.go:356] "Generic (PLEG): container finished" podID="ea6024f6-0d9c-4528-b1d5-895623a932d1" containerID="3aff1f12307a7612a70e72faaffc84721f54186f921a64a9565ac3dbfef2d48c" exitCode=0
Jul 12 00:08:40.396815 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.396357    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42" event={"ID":"ea6024f6-0d9c-4528-b1d5-895623a932d1","Type":"ContainerDied","Data":"3aff1f12307a7612a70e72faaffc84721f54186f921a64a9565ac3dbfef2d48c"}
Jul 12 00:08:40.396815 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.396412    2569 scope.go:117] "RemoveContainer" containerID="7d8d069a2c38b1d015efe4f60b85a9f97230f14b4fe66fedf859d13f8aa64b37"
Jul 12 00:08:40.488880 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.488818    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:08:40.558217 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.558184    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-workload-socket\") pod \"ea6024f6-0d9c-4528-b1d5-895623a932d1\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") "
Jul 12 00:08:40.558382 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.558225    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea6024f6-0d9c-4528-b1d5-895623a932d1-istiod-ca-cert\") pod \"ea6024f6-0d9c-4528-b1d5-895623a932d1\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") "
Jul 12 00:08:40.558382 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.558246    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-credential-socket\") pod \"ea6024f6-0d9c-4528-b1d5-895623a932d1\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") "
Jul 12 00:08:40.558382 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.558323    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-podinfo\") pod \"ea6024f6-0d9c-4528-b1d5-895623a932d1\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") "
Jul 12 00:08:40.558382 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.558350    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-data\") pod \"ea6024f6-0d9c-4528-b1d5-895623a932d1\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") "
Jul 12 00:08:40.558382 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.558377    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-token\") pod \"ea6024f6-0d9c-4528-b1d5-895623a932d1\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") "
Jul 12 00:08:40.558672 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.558412    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbhwd\" (UniqueName: \"kubernetes.io/projected/ea6024f6-0d9c-4528-b1d5-895623a932d1-kube-api-access-sbhwd\") pod \"ea6024f6-0d9c-4528-b1d5-895623a932d1\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") "
Jul 12 00:08:40.558672 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.558471    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-envoy\") pod \"ea6024f6-0d9c-4528-b1d5-895623a932d1\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") "
Jul 12 00:08:40.558672 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.558491    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-workload-certs\") pod \"ea6024f6-0d9c-4528-b1d5-895623a932d1\" (UID: \"ea6024f6-0d9c-4528-b1d5-895623a932d1\") "
Jul 12 00:08:40.558672 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.558595    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea6024f6-0d9c-4528-b1d5-895623a932d1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ea6024f6-0d9c-4528-b1d5-895623a932d1" (UID: "ea6024f6-0d9c-4528-b1d5-895623a932d1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:08:40.558897 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.558820    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ea6024f6-0d9c-4528-b1d5-895623a932d1" (UID: "ea6024f6-0d9c-4528-b1d5-895623a932d1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:08:40.558897 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.558589    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ea6024f6-0d9c-4528-b1d5-895623a932d1" (UID: "ea6024f6-0d9c-4528-b1d5-895623a932d1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:08:40.558897 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.558884    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea6024f6-0d9c-4528-b1d5-895623a932d1-istiod-ca-cert\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:08:40.559045 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.558904    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ea6024f6-0d9c-4528-b1d5-895623a932d1" (UID: "ea6024f6-0d9c-4528-b1d5-895623a932d1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:08:40.559446 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.559400    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-data" (OuterVolumeSpecName: "istio-data") pod "ea6024f6-0d9c-4528-b1d5-895623a932d1" (UID: "ea6024f6-0d9c-4528-b1d5-895623a932d1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:08:40.560652 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.560631    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ea6024f6-0d9c-4528-b1d5-895623a932d1" (UID: "ea6024f6-0d9c-4528-b1d5-895623a932d1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 12 00:08:40.560735 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.560628    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-token" (OuterVolumeSpecName: "istio-token") pod "ea6024f6-0d9c-4528-b1d5-895623a932d1" (UID: "ea6024f6-0d9c-4528-b1d5-895623a932d1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:08:40.561025 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.560997    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ea6024f6-0d9c-4528-b1d5-895623a932d1" (UID: "ea6024f6-0d9c-4528-b1d5-895623a932d1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:08:40.561115 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.561024    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea6024f6-0d9c-4528-b1d5-895623a932d1-kube-api-access-sbhwd" (OuterVolumeSpecName: "kube-api-access-sbhwd") pod "ea6024f6-0d9c-4528-b1d5-895623a932d1" (UID: "ea6024f6-0d9c-4528-b1d5-895623a932d1"). InnerVolumeSpecName "kube-api-access-sbhwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:08:40.659480 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.659444    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-data\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:08:40.659480 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.659474    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-token\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:08:40.659689 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.659487    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbhwd\" (UniqueName: \"kubernetes.io/projected/ea6024f6-0d9c-4528-b1d5-895623a932d1-kube-api-access-sbhwd\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:08:40.659689 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.659501    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-envoy\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:08:40.659689 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.659514    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-workload-certs\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:08:40.659689 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.659526    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-workload-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:08:40.659689 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.659536    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea6024f6-0d9c-4528-b1d5-895623a932d1-credential-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:08:40.659689 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:40.659548    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea6024f6-0d9c-4528-b1d5-895623a932d1-istio-podinfo\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:08:41.402726 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:41.402694    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"
Jul 12 00:08:41.403158 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:41.402699    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42" event={"ID":"ea6024f6-0d9c-4528-b1d5-895623a932d1","Type":"ContainerDied","Data":"de0fbeb953fc2bb91ef76312c3a45f935e0b06ecca6e9ebf94cf2cb5258b43fa"}
Jul 12 00:08:41.403158 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:41.402844    2569 scope.go:117] "RemoveContainer" containerID="3aff1f12307a7612a70e72faaffc84721f54186f921a64a9565ac3dbfef2d48c"
Jul 12 00:08:41.428014 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:41.427960    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"]
Jul 12 00:08:41.429097 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:41.429070    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-7007a3-openshift-default-74794cc87c-74f42"]
Jul 12 00:08:42.983171 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:08:42.983138    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea6024f6-0d9c-4528-b1d5-895623a932d1" path="/var/lib/kubelet/pods/ea6024f6-0d9c-4528-b1d5-895623a932d1/volumes"
Jul 12 00:09:01.367686 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:01.367654    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"]
Jul 12 00:09:01.368111 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:01.367859    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb" podUID="f297935c-26a7-4c5e-8e98-41e45e73efad" containerName="istio-proxy" containerID="cri-o://6781f42961d1241ba22239467380b8483997c9014f7ef1bd2f3f7fa9bf583384" gracePeriod=30
Jul 12 00:09:02.316681 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.316634    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"]
Jul 12 00:09:02.317100 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.317083    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea6024f6-0d9c-4528-b1d5-895623a932d1" containerName="istio-proxy"
Jul 12 00:09:02.317100 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.317101    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea6024f6-0d9c-4528-b1d5-895623a932d1" containerName="istio-proxy"
Jul 12 00:09:02.317200 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.317110    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea6024f6-0d9c-4528-b1d5-895623a932d1" containerName="istio-proxy"
Jul 12 00:09:02.317200 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.317116    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea6024f6-0d9c-4528-b1d5-895623a932d1" containerName="istio-proxy"
Jul 12 00:09:02.317200 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.317132    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1381f12e-957e-4f32-9883-a97016a0f7e2" containerName="revoke-keys"
Jul 12 00:09:02.317200 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.317137    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1381f12e-957e-4f32-9883-a97016a0f7e2" containerName="revoke-keys"
Jul 12 00:09:02.317200 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.317194    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea6024f6-0d9c-4528-b1d5-895623a932d1" containerName="istio-proxy"
Jul 12 00:09:02.317359 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.317204    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1381f12e-957e-4f32-9883-a97016a0f7e2" containerName="revoke-keys"
Jul 12 00:09:02.320240 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.320223    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.323034 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.323008    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-665549-openshift-default-dockercfg-5w6qj\""
Jul 12 00:09:02.330090 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.330058    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"]
Jul 12 00:09:02.456531 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.456495    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-credential-socket\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.456942 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.456534    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-data\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.456942 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.456569    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-workload-socket\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.456942 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.456594    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nddt7\" (UniqueName: \"kubernetes.io/projected/e57b6232-50cb-4c66-8716-644ba5eebcae-kube-api-access-nddt7\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.456942 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.456700    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-token\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.456942 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.456731    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-podinfo\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.456942 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.456758    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-workload-certs\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.456942 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.456875    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-envoy\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.456942 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.456921    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e57b6232-50cb-4c66-8716-644ba5eebcae-istiod-ca-cert\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.558158 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.558116    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-envoy\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.558158 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.558163    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e57b6232-50cb-4c66-8716-644ba5eebcae-istiod-ca-cert\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.558466 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.558205    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-credential-socket\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.558466 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.558225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-data\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.558466 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.558255    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-workload-socket\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.558466 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.558282    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nddt7\" (UniqueName: \"kubernetes.io/projected/e57b6232-50cb-4c66-8716-644ba5eebcae-kube-api-access-nddt7\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.558466 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.558331    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-token\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.558466 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.558355    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-podinfo\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.558466 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.558391    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-workload-certs\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.558813 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.558728    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-credential-socket\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.558913 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.558894    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-workload-certs\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.558994 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.558957    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-data\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.558994 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.558974    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-workload-socket\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.559066 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.559019    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e57b6232-50cb-4c66-8716-644ba5eebcae-istiod-ca-cert\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.560560 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.560542    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-envoy\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.561250 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.561227    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-podinfo\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.566619 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.566590    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nddt7\" (UniqueName: \"kubernetes.io/projected/e57b6232-50cb-4c66-8716-644ba5eebcae-kube-api-access-nddt7\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.566769 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.566726    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-token\") pod \"e2e-api-a-665549-openshift-default-7595f8b977-bb6wq\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") " pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.631809 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.631765    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:02.771926 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.771900    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"]
Jul 12 00:09:02.773927 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:09:02.773899    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode57b6232_50cb_4c66_8716_644ba5eebcae.slice/crio-2afbd75d3ceafa6b005f2806d468095c3d36bad9fe04eda7f4d196490b7c4895 WatchSource:0}: Error finding container 2afbd75d3ceafa6b005f2806d468095c3d36bad9fe04eda7f4d196490b7c4895: Status 404 returned error can't find the container with id 2afbd75d3ceafa6b005f2806d468095c3d36bad9fe04eda7f4d196490b7c4895
Jul 12 00:09:02.776118 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.776073    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:09:02.776188 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.776170    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:09:02.776237 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:02.776221    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:09:03.512806 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:03.512756    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq" event={"ID":"e57b6232-50cb-4c66-8716-644ba5eebcae","Type":"ContainerStarted","Data":"4bf4db21b6620980e207e2d2b232711b1f551dc9fcefec3c147ddaba4ea98613"}
Jul 12 00:09:03.513206 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:03.512828    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq" event={"ID":"e57b6232-50cb-4c66-8716-644ba5eebcae","Type":"ContainerStarted","Data":"2afbd75d3ceafa6b005f2806d468095c3d36bad9fe04eda7f4d196490b7c4895"}
Jul 12 00:09:03.533365 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:03.533316    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq" podStartSLOduration=1.5333005069999999 podStartE2EDuration="1.533300507s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:09:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:09:03.531037176 +0000 UTC m=+2273.109129951" watchObservedRunningTime="2026-07-12 00:09:03.533300507 +0000 UTC m=+2273.111393282"
Jul 12 00:09:03.631986 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:03.631944    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:03.637321 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:03.637288    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:04.517527 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:04.517499    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:04.518558 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:04.518538    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:05.649408 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:05.649372    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"]
Jul 12 00:09:05.650213 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:05.650188    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea6024f6-0d9c-4528-b1d5-895623a932d1" containerName="istio-proxy"
Jul 12 00:09:05.653471 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:05.653451    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"
Jul 12 00:09:05.655760 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:05.655737    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-665549-serving-cert\""
Jul 12 00:09:05.665887 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:05.665861    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"]
Jul 12 00:09:05.788956 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:05.788916    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqxbm\" (UniqueName: \"kubernetes.io/projected/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-kube-api-access-sqxbm\") pod \"maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9\" (UID: \"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"
Jul 12 00:09:05.789160 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:05.788964    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-maas-api-tls\") pod \"maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9\" (UID: \"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"
Jul 12 00:09:05.789160 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:05.788989    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-openshift-service-ca\") pod \"maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9\" (UID: \"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"
Jul 12 00:09:05.890588 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:05.890546    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqxbm\" (UniqueName: \"kubernetes.io/projected/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-kube-api-access-sqxbm\") pod \"maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9\" (UID: \"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"
Jul 12 00:09:05.890588 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:05.890594    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-maas-api-tls\") pod \"maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9\" (UID: \"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"
Jul 12 00:09:05.890841 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:05.890615    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-openshift-service-ca\") pod \"maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9\" (UID: \"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"
Jul 12 00:09:05.890841 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:09:05.890707    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-665549-serving-cert: secret "maas-api-e2e-api-a-665549-serving-cert" not found
Jul 12 00:09:05.890841 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:09:05.890783    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-maas-api-tls podName:a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0 nodeName:}" failed. No retries permitted until 2026-07-12 00:09:06.3907681 +0000 UTC m=+2275.968860854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-maas-api-tls") pod "maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9" (UID: "a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0") : secret "maas-api-e2e-api-a-665549-serving-cert" not found
Jul 12 00:09:05.891222 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:05.891203    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-openshift-service-ca\") pod \"maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9\" (UID: \"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"
Jul 12 00:09:05.903758 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:05.903682    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqxbm\" (UniqueName: \"kubernetes.io/projected/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-kube-api-access-sqxbm\") pod \"maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9\" (UID: \"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"
Jul 12 00:09:06.395392 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.395363    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-maas-api-tls\") pod \"maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9\" (UID: \"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"
Jul 12 00:09:06.397824 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.397801    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-maas-api-tls\") pod \"maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9\" (UID: \"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"
Jul 12 00:09:06.532666 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.532626    2569 generic.go:356] "Generic (PLEG): container finished" podID="f297935c-26a7-4c5e-8e98-41e45e73efad" containerID="6781f42961d1241ba22239467380b8483997c9014f7ef1bd2f3f7fa9bf583384" exitCode=0
Jul 12 00:09:06.533037 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.533006    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb" event={"ID":"f297935c-26a7-4c5e-8e98-41e45e73efad","Type":"ContainerDied","Data":"6781f42961d1241ba22239467380b8483997c9014f7ef1bd2f3f7fa9bf583384"}
Jul 12 00:09:06.533154 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.533091    2569 scope.go:117] "RemoveContainer" containerID="c613d8e02224199a7c46b0bd378e2f548d77971219bfbbd24c9ca424e0788c88"
Jul 12 00:09:06.566340 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.566295    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"
Jul 12 00:09:06.613793 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.613767    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:09:06.708499 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.708473    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"]
Jul 12 00:09:06.709908 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:09:06.709886    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda68228c5_9f5c_4a15_bfe2_6fbbd8bc80d0.slice/crio-dfe9d369153bae20d747b010f804df8f7dea33eef3076e4a24d6a7faac6cddf6 WatchSource:0}: Error finding container dfe9d369153bae20d747b010f804df8f7dea33eef3076e4a24d6a7faac6cddf6: Status 404 returned error can't find the container with id dfe9d369153bae20d747b010f804df8f7dea33eef3076e4a24d6a7faac6cddf6
Jul 12 00:09:06.711338 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.711317    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 12 00:09:06.798512 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.798480    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-credential-socket\") pod \"f297935c-26a7-4c5e-8e98-41e45e73efad\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") "
Jul 12 00:09:06.798706 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.798545    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f297935c-26a7-4c5e-8e98-41e45e73efad-istiod-ca-cert\") pod \"f297935c-26a7-4c5e-8e98-41e45e73efad\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") "
Jul 12 00:09:06.798706 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.798567    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-data\") pod \"f297935c-26a7-4c5e-8e98-41e45e73efad\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") "
Jul 12 00:09:06.798706 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.798592    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-workload-certs\") pod \"f297935c-26a7-4c5e-8e98-41e45e73efad\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") "
Jul 12 00:09:06.798706 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.798649    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-podinfo\") pod \"f297935c-26a7-4c5e-8e98-41e45e73efad\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") "
Jul 12 00:09:06.798706 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.798688    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-envoy\") pod \"f297935c-26a7-4c5e-8e98-41e45e73efad\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") "
Jul 12 00:09:06.798963 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.798806    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f297935c-26a7-4c5e-8e98-41e45e73efad" (UID: "f297935c-26a7-4c5e-8e98-41e45e73efad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:09:06.798963 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.798852    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-workload-socket\") pod \"f297935c-26a7-4c5e-8e98-41e45e73efad\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") "
Jul 12 00:09:06.798963 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.798910    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkjmz\" (UniqueName: \"kubernetes.io/projected/f297935c-26a7-4c5e-8e98-41e45e73efad-kube-api-access-zkjmz\") pod \"f297935c-26a7-4c5e-8e98-41e45e73efad\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") "
Jul 12 00:09:06.798963 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.798942    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-token\") pod \"f297935c-26a7-4c5e-8e98-41e45e73efad\" (UID: \"f297935c-26a7-4c5e-8e98-41e45e73efad\") "
Jul 12 00:09:06.799185 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.798965    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f297935c-26a7-4c5e-8e98-41e45e73efad" (UID: "f297935c-26a7-4c5e-8e98-41e45e73efad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:09:06.799185 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.798998    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f297935c-26a7-4c5e-8e98-41e45e73efad-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f297935c-26a7-4c5e-8e98-41e45e73efad" (UID: "f297935c-26a7-4c5e-8e98-41e45e73efad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:09:06.799185 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.799126    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f297935c-26a7-4c5e-8e98-41e45e73efad" (UID: "f297935c-26a7-4c5e-8e98-41e45e73efad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:09:06.799353 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.799278    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f297935c-26a7-4c5e-8e98-41e45e73efad-istiod-ca-cert\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:06.799353 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.799298    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-workload-certs\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:06.799353 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.799310    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-workload-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:06.799353 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.799319    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-credential-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:06.799579 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.799376    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-data" (OuterVolumeSpecName: "istio-data") pod "f297935c-26a7-4c5e-8e98-41e45e73efad" (UID: "f297935c-26a7-4c5e-8e98-41e45e73efad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:09:06.800984 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.800954    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f297935c-26a7-4c5e-8e98-41e45e73efad" (UID: "f297935c-26a7-4c5e-8e98-41e45e73efad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 12 00:09:06.800984 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.800961    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f297935c-26a7-4c5e-8e98-41e45e73efad" (UID: "f297935c-26a7-4c5e-8e98-41e45e73efad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:09:06.801129 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.801019    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f297935c-26a7-4c5e-8e98-41e45e73efad-kube-api-access-zkjmz" (OuterVolumeSpecName: "kube-api-access-zkjmz") pod "f297935c-26a7-4c5e-8e98-41e45e73efad" (UID: "f297935c-26a7-4c5e-8e98-41e45e73efad"). InnerVolumeSpecName "kube-api-access-zkjmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:09:06.801129 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.801037    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-token" (OuterVolumeSpecName: "istio-token") pod "f297935c-26a7-4c5e-8e98-41e45e73efad" (UID: "f297935c-26a7-4c5e-8e98-41e45e73efad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:09:06.900485 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.900448    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-data\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:06.900485 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.900475    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-podinfo\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:06.900485 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.900483    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-envoy\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:06.900485 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.900492    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkjmz\" (UniqueName: \"kubernetes.io/projected/f297935c-26a7-4c5e-8e98-41e45e73efad-kube-api-access-zkjmz\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:06.900838 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:06.900502    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f297935c-26a7-4c5e-8e98-41e45e73efad-istio-token\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:07.539811 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:07.539766    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9" event={"ID":"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0","Type":"ContainerStarted","Data":"218333f568bf8a778b2ec5bebaf4e2d33fecee931cad7372cd49211a6d85b86f"}
Jul 12 00:09:07.540019 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:07.539833    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9" event={"ID":"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0","Type":"ContainerStarted","Data":"dfe9d369153bae20d747b010f804df8f7dea33eef3076e4a24d6a7faac6cddf6"}
Jul 12 00:09:07.540019 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:07.539862    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"
Jul 12 00:09:07.541109 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:07.541090    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb" event={"ID":"f297935c-26a7-4c5e-8e98-41e45e73efad","Type":"ContainerDied","Data":"dacb3526946049980004dd6143663c37cebf9906c61a2f9449c3bbd230813e0c"}
Jul 12 00:09:07.541179 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:07.541098    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"
Jul 12 00:09:07.541271 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:07.541131    2569 scope.go:117] "RemoveContainer" containerID="6781f42961d1241ba22239467380b8483997c9014f7ef1bd2f3f7fa9bf583384"
Jul 12 00:09:07.558361 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:07.558277    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9" podStartSLOduration=1.96294213 podStartE2EDuration="2.558263571s" totalImagesPullingTime="595.321441ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:09:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:09:07.555885889 +0000 UTC m=+2277.133978665" watchObservedRunningTime="2026-07-12 00:09:07.558263571 +0000 UTC m=+2277.136356346"
Jul 12 00:09:07.571893 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:07.571866    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"]
Jul 12 00:09:07.575037 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:07.575014    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ad0409-openshift-default-6d7c5c94f9-kclrb"]
Jul 12 00:09:08.982807 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:08.982772    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f297935c-26a7-4c5e-8e98-41e45e73efad" path="/var/lib/kubelet/pods/f297935c-26a7-4c5e-8e98-41e45e73efad/volumes"
Jul 12 00:09:10.860344 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:10.860308    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"]
Jul 12 00:09:10.861257 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:10.861233    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f297935c-26a7-4c5e-8e98-41e45e73efad" containerName="istio-proxy"
Jul 12 00:09:10.861257 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:10.861258    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f297935c-26a7-4c5e-8e98-41e45e73efad" containerName="istio-proxy"
Jul 12 00:09:10.861527 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:10.861302    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f297935c-26a7-4c5e-8e98-41e45e73efad" containerName="istio-proxy"
Jul 12 00:09:10.861527 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:10.861312    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f297935c-26a7-4c5e-8e98-41e45e73efad" containerName="istio-proxy"
Jul 12 00:09:10.861527 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:10.861400    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f297935c-26a7-4c5e-8e98-41e45e73efad" containerName="istio-proxy"
Jul 12 00:09:10.865945 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:10.865918    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:10.868794 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:10.868759    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a58284-openshift-default-dockercfg-xctrm\""
Jul 12 00:09:10.876882 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:10.876769    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"]
Jul 12 00:09:10.934315 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:10.934275    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-credential-socket\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:10.934315 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:10.934314    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-podinfo\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:10.934613 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:10.934352    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-envoy\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:10.934613 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:10.934446    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-698q5\" (UniqueName: \"kubernetes.io/projected/0c791880-8074-41a0-8f84-24fa41f2b0c6-kube-api-access-698q5\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:10.934613 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:10.934491    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-token\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:10.934613 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:10.934519    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-data\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:10.934613 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:10.934539    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-workload-socket\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:10.934613 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:10.934559    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-workload-certs\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:10.934613 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:10.934597    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c791880-8074-41a0-8f84-24fa41f2b0c6-istiod-ca-cert\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.035284 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.035247    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-workload-certs\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.035507 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.035301    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c791880-8074-41a0-8f84-24fa41f2b0c6-istiod-ca-cert\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.035507 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.035328    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-credential-socket\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.035507 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.035346    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-podinfo\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.035507 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.035370    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-envoy\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.035507 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.035399    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-698q5\" (UniqueName: \"kubernetes.io/projected/0c791880-8074-41a0-8f84-24fa41f2b0c6-kube-api-access-698q5\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.035789 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.035539    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-token\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.035789 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.035589    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-data\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.035789 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.035619    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-workload-socket\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.035789 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.035766    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-workload-certs\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.036080 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.036056    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-credential-socket\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.036156 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.036084    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c791880-8074-41a0-8f84-24fa41f2b0c6-istiod-ca-cert\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.036220 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.036169    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-data\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.036309 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.036291    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-workload-socket\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.038043 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.038015    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-envoy\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.038414 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.038395    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-podinfo\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.045314 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.045281    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-token\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.045468 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.045419    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-698q5\" (UniqueName: \"kubernetes.io/projected/0c791880-8074-41a0-8f84-24fa41f2b0c6-kube-api-access-698q5\") pod \"e2e-api-b-a58284-openshift-default-57667b9977-fk65r\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") " pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.183820 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.183463    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:11.353733 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.353706    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"]
Jul 12 00:09:11.354724 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:09:11.354701    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c791880_8074_41a0_8f84_24fa41f2b0c6.slice/crio-cb784443e9131b3e5f6cb8f464c5ef74779127ada6f05522915bb4ae0e156a61 WatchSource:0}: Error finding container cb784443e9131b3e5f6cb8f464c5ef74779127ada6f05522915bb4ae0e156a61: Status 404 returned error can't find the container with id cb784443e9131b3e5f6cb8f464c5ef74779127ada6f05522915bb4ae0e156a61
Jul 12 00:09:11.357059 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.357023    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:09:11.357169 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.357108    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:09:11.357169 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.357143    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:09:11.569717 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.569673    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r" event={"ID":"0c791880-8074-41a0-8f84-24fa41f2b0c6","Type":"ContainerStarted","Data":"1abf10c214ae00fab0a8ee784c1fc3991784464fa0d10865743a43d1ba2affa1"}
Jul 12 00:09:11.569925 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.569747    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r" event={"ID":"0c791880-8074-41a0-8f84-24fa41f2b0c6","Type":"ContainerStarted","Data":"cb784443e9131b3e5f6cb8f464c5ef74779127ada6f05522915bb4ae0e156a61"}
Jul 12 00:09:11.590625 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:11.590562    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r" podStartSLOduration=1.590539873 podStartE2EDuration="1.590539873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:09:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:09:11.588954035 +0000 UTC m=+2281.167046816" watchObservedRunningTime="2026-07-12 00:09:11.590539873 +0000 UTC m=+2281.168632650"
Jul 12 00:09:12.185296 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:12.185261    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:12.190633 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:12.190606    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:12.595382 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:12.595339    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:12.596544 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:12.596522    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:13.553535 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:13.553504    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"
Jul 12 00:09:18.628849 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:18.628816    2569 generic.go:356] "Generic (PLEG): container finished" podID="e57b6232-50cb-4c66-8716-644ba5eebcae" containerID="4bf4db21b6620980e207e2d2b232711b1f551dc9fcefec3c147ddaba4ea98613" exitCode=0
Jul 12 00:09:18.629268 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:18.628891    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq" event={"ID":"e57b6232-50cb-4c66-8716-644ba5eebcae","Type":"ContainerDied","Data":"4bf4db21b6620980e207e2d2b232711b1f551dc9fcefec3c147ddaba4ea98613"}
Jul 12 00:09:18.629316 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:18.629287    2569 scope.go:117] "RemoveContainer" containerID="4bf4db21b6620980e207e2d2b232711b1f551dc9fcefec3c147ddaba4ea98613"
Jul 12 00:09:18.630353 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:18.630326    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:09:18.630456 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:18.630390    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:09:18.630501 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:18.630459    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:09:19.518198 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:19.518152    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:19.635953 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:19.635913    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq" event={"ID":"e57b6232-50cb-4c66-8716-644ba5eebcae","Type":"ContainerStarted","Data":"632c030bbd984e9ba42e42a9350c4d7da5b1c1aee814a0fc273195b6e59c1f87"}
Jul 12 00:09:20.632848 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:20.632818    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:20.633894 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:20.633866    2569 patch_prober.go:28] interesting pod/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 12 00:09:20.634030 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:20.633914    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq" podUID="e57b6232-50cb-4c66-8716-644ba5eebcae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 12 00:09:21.632861 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:21.632820    2569 patch_prober.go:28] interesting pod/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 12 00:09:21.633259 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:21.632881    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq" podUID="e57b6232-50cb-4c66-8716-644ba5eebcae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 12 00:09:22.633101 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:22.633061    2569 patch_prober.go:28] interesting pod/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 12 00:09:22.633535 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:22.633134    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq" podUID="e57b6232-50cb-4c66-8716-644ba5eebcae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 12 00:09:23.637040 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:23.637010    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:23.637472 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:23.637275    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:23.637880 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:23.637865    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:28.637640 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:28.637588    2569 patch_prober.go:28] interesting pod/e2e-api-b-a58284-openshift-default-57667b9977-fk65r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 12 00:09:28.638174 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:28.637648    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r" podUID="0c791880-8074-41a0-8f84-24fa41f2b0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 12 00:09:29.694262 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:29.694229    2569 generic.go:356] "Generic (PLEG): container finished" podID="0c791880-8074-41a0-8f84-24fa41f2b0c6" containerID="1abf10c214ae00fab0a8ee784c1fc3991784464fa0d10865743a43d1ba2affa1" exitCode=0
Jul 12 00:09:29.694711 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:29.694304    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r" event={"ID":"0c791880-8074-41a0-8f84-24fa41f2b0c6","Type":"ContainerDied","Data":"1abf10c214ae00fab0a8ee784c1fc3991784464fa0d10865743a43d1ba2affa1"}
Jul 12 00:09:29.694801 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:29.694726    2569 scope.go:117] "RemoveContainer" containerID="1abf10c214ae00fab0a8ee784c1fc3991784464fa0d10865743a43d1ba2affa1"
Jul 12 00:09:29.695729 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:29.695696    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:09:29.695849 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:29.695763    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:09:29.695849 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:29.695796    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:09:30.701277 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:30.701239    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r" event={"ID":"0c791880-8074-41a0-8f84-24fa41f2b0c6","Type":"ContainerStarted","Data":"f1756e53c5f278d0c634e0f9425cb3c18cdff06cff2b3db356455b3553b4bb0d"}
Jul 12 00:09:31.184777 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:31.184726    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:31.184994 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:31.184954    2569 patch_prober.go:28] interesting pod/e2e-api-b-a58284-openshift-default-57667b9977-fk65r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 12 00:09:31.185073 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:31.185002    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r" podUID="0c791880-8074-41a0-8f84-24fa41f2b0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 12 00:09:32.184658 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:32.184610    2569 patch_prober.go:28] interesting pod/e2e-api-b-a58284-openshift-default-57667b9977-fk65r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 12 00:09:32.185118 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:32.184683    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r" podUID="0c791880-8074-41a0-8f84-24fa41f2b0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 12 00:09:33.185021 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:33.184919    2569 patch_prober.go:28] interesting pod/e2e-api-b-a58284-openshift-default-57667b9977-fk65r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 12 00:09:33.185021 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:33.185000    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r" podUID="0c791880-8074-41a0-8f84-24fa41f2b0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 12 00:09:34.185226 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:34.185182    2569 patch_prober.go:28] interesting pod/e2e-api-b-a58284-openshift-default-57667b9977-fk65r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 12 00:09:34.185664 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:34.185250    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r" podUID="0c791880-8074-41a0-8f84-24fa41f2b0c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 12 00:09:35.187647 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:35.187613    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:35.188036 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:35.187954    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:35.188579 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:35.188560    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:09:37.022962 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.022920    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"]
Jul 12 00:09:37.023476 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.023192    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9" podUID="a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0" containerName="maas-api" containerID="cri-o://218333f568bf8a778b2ec5bebaf4e2d33fecee931cad7372cd49211a6d85b86f" gracePeriod=30
Jul 12 00:09:37.283046 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.282985    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"
Jul 12 00:09:37.384237 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.384207    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-maas-api-tls\") pod \"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0\" (UID: \"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0\") "
Jul 12 00:09:37.384415 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.384286    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-openshift-service-ca\") pod \"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0\" (UID: \"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0\") "
Jul 12 00:09:37.384415 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.384329    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqxbm\" (UniqueName: \"kubernetes.io/projected/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-kube-api-access-sqxbm\") pod \"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0\" (UID: \"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0\") "
Jul 12 00:09:37.384717 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.384688    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0" (UID: "a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:09:37.386411 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.386385    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0" (UID: "a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 12 00:09:37.386493 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.386477    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-kube-api-access-sqxbm" (OuterVolumeSpecName: "kube-api-access-sqxbm") pod "a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0" (UID: "a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0"). InnerVolumeSpecName "kube-api-access-sqxbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:09:37.485404 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.485369    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-maas-api-tls\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:37.485404 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.485399    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-openshift-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:37.485404 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.485410    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqxbm\" (UniqueName: \"kubernetes.io/projected/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0-kube-api-access-sqxbm\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:37.737139 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.737103    2569 generic.go:356] "Generic (PLEG): container finished" podID="a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0" containerID="218333f568bf8a778b2ec5bebaf4e2d33fecee931cad7372cd49211a6d85b86f" exitCode=0
Jul 12 00:09:37.737341 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.737150    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"
Jul 12 00:09:37.737341 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.737168    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9" event={"ID":"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0","Type":"ContainerDied","Data":"218333f568bf8a778b2ec5bebaf4e2d33fecee931cad7372cd49211a6d85b86f"}
Jul 12 00:09:37.737341 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.737211    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9" event={"ID":"a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0","Type":"ContainerDied","Data":"dfe9d369153bae20d747b010f804df8f7dea33eef3076e4a24d6a7faac6cddf6"}
Jul 12 00:09:37.737341 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.737232    2569 scope.go:117] "RemoveContainer" containerID="218333f568bf8a778b2ec5bebaf4e2d33fecee931cad7372cd49211a6d85b86f"
Jul 12 00:09:37.746166 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.746146    2569 scope.go:117] "RemoveContainer" containerID="218333f568bf8a778b2ec5bebaf4e2d33fecee931cad7372cd49211a6d85b86f"
Jul 12 00:09:37.746441 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:09:37.746400    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"218333f568bf8a778b2ec5bebaf4e2d33fecee931cad7372cd49211a6d85b86f\": container with ID starting with 218333f568bf8a778b2ec5bebaf4e2d33fecee931cad7372cd49211a6d85b86f not found: ID does not exist" containerID="218333f568bf8a778b2ec5bebaf4e2d33fecee931cad7372cd49211a6d85b86f"
Jul 12 00:09:37.746490 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.746450    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"218333f568bf8a778b2ec5bebaf4e2d33fecee931cad7372cd49211a6d85b86f"} err="failed to get container status \"218333f568bf8a778b2ec5bebaf4e2d33fecee931cad7372cd49211a6d85b86f\": rpc error: code = NotFound desc = could not find container \"218333f568bf8a778b2ec5bebaf4e2d33fecee931cad7372cd49211a6d85b86f\": container with ID starting with 218333f568bf8a778b2ec5bebaf4e2d33fecee931cad7372cd49211a6d85b86f not found: ID does not exist"
Jul 12 00:09:37.758388 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.758350    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"]
Jul 12 00:09:37.762864 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:37.762837    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-665549-6f87d7d6b9-nl4k9"]
Jul 12 00:09:38.983004 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:38.982969    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0" path="/var/lib/kubelet/pods/a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0/volumes"
Jul 12 00:09:52.076973 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:52.076932    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"]
Jul 12 00:09:52.077529 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:52.077199    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq" podUID="e57b6232-50cb-4c66-8716-644ba5eebcae" containerName="istio-proxy" containerID="cri-o://632c030bbd984e9ba42e42a9350c4d7da5b1c1aee814a0fc273195b6e59c1f87" gracePeriod=30
Jul 12 00:09:52.462157 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:52.462059    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a58284-9wtdl"]
Jul 12 00:09:52.462540 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:52.462522    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0" containerName="maas-api"
Jul 12 00:09:52.462640 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:52.462541    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0" containerName="maas-api"
Jul 12 00:09:52.462698 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:52.462652    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f297935c-26a7-4c5e-8e98-41e45e73efad" containerName="istio-proxy"
Jul 12 00:09:52.462698 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:52.462671    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a68228c5-9f5c-4a15-bfe2-6fbbd8bc80d0" containerName="maas-api"
Jul 12 00:09:52.465923 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:52.465900    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a58284-9wtdl"
Jul 12 00:09:52.468408 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:52.468383    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\""
Jul 12 00:09:52.473027 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:52.472998    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a58284-9wtdl"]
Jul 12 00:09:52.623048 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:52.623004    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac2bf81d-1eec-4a10-bfa1-3088a64e15a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a58284-9wtdl\" (UID: \"ac2bf81d-1eec-4a10-bfa1-3088a64e15a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a58284-9wtdl"
Jul 12 00:09:52.724590 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:52.724492    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac2bf81d-1eec-4a10-bfa1-3088a64e15a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a58284-9wtdl\" (UID: \"ac2bf81d-1eec-4a10-bfa1-3088a64e15a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a58284-9wtdl"
Jul 12 00:09:52.725141 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:52.725122    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac2bf81d-1eec-4a10-bfa1-3088a64e15a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a58284-9wtdl\" (UID: \"ac2bf81d-1eec-4a10-bfa1-3088a64e15a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a58284-9wtdl"
Jul 12 00:09:52.778880 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:52.778845    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a58284-9wtdl"
Jul 12 00:09:52.912577 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:52.912553    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a58284-9wtdl"]
Jul 12 00:09:52.914930 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:09:52.914889    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac2bf81d_1eec_4a10_bfa1_3088a64e15a0.slice/crio-b8f7f531bca4fbbd414a07f1ae8a2011df412f2cecefbb195b6451898593d4ce WatchSource:0}: Error finding container b8f7f531bca4fbbd414a07f1ae8a2011df412f2cecefbb195b6451898593d4ce: Status 404 returned error can't find the container with id b8f7f531bca4fbbd414a07f1ae8a2011df412f2cecefbb195b6451898593d4ce
Jul 12 00:09:53.639106 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:53.639071    2569 patch_prober.go:28] interesting pod/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 12 00:09:53.639451 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:53.639135    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq" podUID="e57b6232-50cb-4c66-8716-644ba5eebcae" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 12 00:09:53.817980 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:53.817938    2569 generic.go:356] "Generic (PLEG): container finished" podID="ac2bf81d-1eec-4a10-bfa1-3088a64e15a0" containerID="8fa3efc78dfe8154ebc0acf67fd79f0957d7e6b28e6ddf00f3764342b21a796d" exitCode=0
Jul 12 00:09:53.818185 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:53.817994    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a58284-9wtdl" event={"ID":"ac2bf81d-1eec-4a10-bfa1-3088a64e15a0","Type":"ContainerDied","Data":"8fa3efc78dfe8154ebc0acf67fd79f0957d7e6b28e6ddf00f3764342b21a796d"}
Jul 12 00:09:53.818185 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:53.818050    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a58284-9wtdl" event={"ID":"ac2bf81d-1eec-4a10-bfa1-3088a64e15a0","Type":"ContainerStarted","Data":"b8f7f531bca4fbbd414a07f1ae8a2011df412f2cecefbb195b6451898593d4ce"}
Jul 12 00:09:54.953910 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:54.953887    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a58284-9wtdl"
Jul 12 00:09:55.146181 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:55.146094    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac2bf81d-1eec-4a10-bfa1-3088a64e15a0-maas-api-service-ca\") pod \"ac2bf81d-1eec-4a10-bfa1-3088a64e15a0\" (UID: \"ac2bf81d-1eec-4a10-bfa1-3088a64e15a0\") "
Jul 12 00:09:55.146489 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:55.146464    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac2bf81d-1eec-4a10-bfa1-3088a64e15a0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ac2bf81d-1eec-4a10-bfa1-3088a64e15a0" (UID: "ac2bf81d-1eec-4a10-bfa1-3088a64e15a0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:09:55.247088 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:55.247053    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac2bf81d-1eec-4a10-bfa1-3088a64e15a0-maas-api-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:55.829501 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:55.829456    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a58284-9wtdl"
Jul 12 00:09:55.829501 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:55.829487    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a58284-9wtdl" event={"ID":"ac2bf81d-1eec-4a10-bfa1-3088a64e15a0","Type":"ContainerDied","Data":"b8f7f531bca4fbbd414a07f1ae8a2011df412f2cecefbb195b6451898593d4ce"}
Jul 12 00:09:55.829753 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:55.829540    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8f7f531bca4fbbd414a07f1ae8a2011df412f2cecefbb195b6451898593d4ce"
Jul 12 00:09:57.338361 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.338298    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:57.364656 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.364627    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-data\") pod \"e57b6232-50cb-4c66-8716-644ba5eebcae\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") "
Jul 12 00:09:57.364805 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.364662    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-workload-socket\") pod \"e57b6232-50cb-4c66-8716-644ba5eebcae\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") "
Jul 12 00:09:57.364805 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.364695    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-workload-certs\") pod \"e57b6232-50cb-4c66-8716-644ba5eebcae\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") "
Jul 12 00:09:57.364805 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.364730    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e57b6232-50cb-4c66-8716-644ba5eebcae-istiod-ca-cert\") pod \"e57b6232-50cb-4c66-8716-644ba5eebcae\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") "
Jul 12 00:09:57.364805 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.364767    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-credential-socket\") pod \"e57b6232-50cb-4c66-8716-644ba5eebcae\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") "
Jul 12 00:09:57.365039 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.364813    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-token\") pod \"e57b6232-50cb-4c66-8716-644ba5eebcae\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") "
Jul 12 00:09:57.365039 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.364859    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nddt7\" (UniqueName: \"kubernetes.io/projected/e57b6232-50cb-4c66-8716-644ba5eebcae-kube-api-access-nddt7\") pod \"e57b6232-50cb-4c66-8716-644ba5eebcae\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") "
Jul 12 00:09:57.365039 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.364882    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-podinfo\") pod \"e57b6232-50cb-4c66-8716-644ba5eebcae\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") "
Jul 12 00:09:57.365039 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.364927    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-envoy\") pod \"e57b6232-50cb-4c66-8716-644ba5eebcae\" (UID: \"e57b6232-50cb-4c66-8716-644ba5eebcae\") "
Jul 12 00:09:57.365039 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.364968    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e57b6232-50cb-4c66-8716-644ba5eebcae" (UID: "e57b6232-50cb-4c66-8716-644ba5eebcae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:09:57.365306 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.365048    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e57b6232-50cb-4c66-8716-644ba5eebcae" (UID: "e57b6232-50cb-4c66-8716-644ba5eebcae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:09:57.365306 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.365198    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-workload-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:57.365306 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.365215    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-workload-certs\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:57.365485 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.365444    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e57b6232-50cb-4c66-8716-644ba5eebcae" (UID: "e57b6232-50cb-4c66-8716-644ba5eebcae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:09:57.365628 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.365581    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e57b6232-50cb-4c66-8716-644ba5eebcae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e57b6232-50cb-4c66-8716-644ba5eebcae" (UID: "e57b6232-50cb-4c66-8716-644ba5eebcae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:09:57.366095 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.366067    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-data" (OuterVolumeSpecName: "istio-data") pod "e57b6232-50cb-4c66-8716-644ba5eebcae" (UID: "e57b6232-50cb-4c66-8716-644ba5eebcae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:09:57.367394 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.367364    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e57b6232-50cb-4c66-8716-644ba5eebcae-kube-api-access-nddt7" (OuterVolumeSpecName: "kube-api-access-nddt7") pod "e57b6232-50cb-4c66-8716-644ba5eebcae" (UID: "e57b6232-50cb-4c66-8716-644ba5eebcae"). InnerVolumeSpecName "kube-api-access-nddt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:09:57.367566 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.367391    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-token" (OuterVolumeSpecName: "istio-token") pod "e57b6232-50cb-4c66-8716-644ba5eebcae" (UID: "e57b6232-50cb-4c66-8716-644ba5eebcae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:09:57.367566 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.367534    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e57b6232-50cb-4c66-8716-644ba5eebcae" (UID: "e57b6232-50cb-4c66-8716-644ba5eebcae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:09:57.367668 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.367651    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e57b6232-50cb-4c66-8716-644ba5eebcae" (UID: "e57b6232-50cb-4c66-8716-644ba5eebcae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 12 00:09:57.466242 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.466204    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e57b6232-50cb-4c66-8716-644ba5eebcae-istiod-ca-cert\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:57.466242 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.466233    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-credential-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:57.466242 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.466242    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-token\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:57.466242 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.466253    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nddt7\" (UniqueName: \"kubernetes.io/projected/e57b6232-50cb-4c66-8716-644ba5eebcae-kube-api-access-nddt7\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:57.466552 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.466261    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-podinfo\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:57.466552 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.466270    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-envoy\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:57.466552 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.466277    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e57b6232-50cb-4c66-8716-644ba5eebcae-istio-data\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:09:57.840537 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.840509    2569 generic.go:356] "Generic (PLEG): container finished" podID="e57b6232-50cb-4c66-8716-644ba5eebcae" containerID="632c030bbd984e9ba42e42a9350c4d7da5b1c1aee814a0fc273195b6e59c1f87" exitCode=0
Jul 12 00:09:57.840748 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.840536    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq" event={"ID":"e57b6232-50cb-4c66-8716-644ba5eebcae","Type":"ContainerDied","Data":"632c030bbd984e9ba42e42a9350c4d7da5b1c1aee814a0fc273195b6e59c1f87"}
Jul 12 00:09:57.840748 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.840567    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"
Jul 12 00:09:57.840748 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.840604    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq" event={"ID":"e57b6232-50cb-4c66-8716-644ba5eebcae","Type":"ContainerDied","Data":"2afbd75d3ceafa6b005f2806d468095c3d36bad9fe04eda7f4d196490b7c4895"}
Jul 12 00:09:57.840748 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.840638    2569 scope.go:117] "RemoveContainer" containerID="632c030bbd984e9ba42e42a9350c4d7da5b1c1aee814a0fc273195b6e59c1f87"
Jul 12 00:09:57.850199 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.850184    2569 scope.go:117] "RemoveContainer" containerID="4bf4db21b6620980e207e2d2b232711b1f551dc9fcefec3c147ddaba4ea98613"
Jul 12 00:09:57.857805 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.857786    2569 scope.go:117] "RemoveContainer" containerID="632c030bbd984e9ba42e42a9350c4d7da5b1c1aee814a0fc273195b6e59c1f87"
Jul 12 00:09:57.858055 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:09:57.858036    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"632c030bbd984e9ba42e42a9350c4d7da5b1c1aee814a0fc273195b6e59c1f87\": container with ID starting with 632c030bbd984e9ba42e42a9350c4d7da5b1c1aee814a0fc273195b6e59c1f87 not found: ID does not exist" containerID="632c030bbd984e9ba42e42a9350c4d7da5b1c1aee814a0fc273195b6e59c1f87"
Jul 12 00:09:57.858102 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.858063    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"632c030bbd984e9ba42e42a9350c4d7da5b1c1aee814a0fc273195b6e59c1f87"} err="failed to get container status \"632c030bbd984e9ba42e42a9350c4d7da5b1c1aee814a0fc273195b6e59c1f87\": rpc error: code = NotFound desc = could not find container \"632c030bbd984e9ba42e42a9350c4d7da5b1c1aee814a0fc273195b6e59c1f87\": container with ID starting with 632c030bbd984e9ba42e42a9350c4d7da5b1c1aee814a0fc273195b6e59c1f87 not found: ID does not exist"
Jul 12 00:09:57.858102 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.858081    2569 scope.go:117] "RemoveContainer" containerID="4bf4db21b6620980e207e2d2b232711b1f551dc9fcefec3c147ddaba4ea98613"
Jul 12 00:09:57.858303 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:09:57.858287    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bf4db21b6620980e207e2d2b232711b1f551dc9fcefec3c147ddaba4ea98613\": container with ID starting with 4bf4db21b6620980e207e2d2b232711b1f551dc9fcefec3c147ddaba4ea98613 not found: ID does not exist" containerID="4bf4db21b6620980e207e2d2b232711b1f551dc9fcefec3c147ddaba4ea98613"
Jul 12 00:09:57.858348 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.858306    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bf4db21b6620980e207e2d2b232711b1f551dc9fcefec3c147ddaba4ea98613"} err="failed to get container status \"4bf4db21b6620980e207e2d2b232711b1f551dc9fcefec3c147ddaba4ea98613\": rpc error: code = NotFound desc = could not find container \"4bf4db21b6620980e207e2d2b232711b1f551dc9fcefec3c147ddaba4ea98613\": container with ID starting with 4bf4db21b6620980e207e2d2b232711b1f551dc9fcefec3c147ddaba4ea98613 not found: ID does not exist"
Jul 12 00:09:57.862808 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.862782    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"]
Jul 12 00:09:57.867215 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:57.867193    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-665549-openshift-default-7595f8b977-bb6wq"]
Jul 12 00:09:58.984230 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:09:58.984200    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e57b6232-50cb-4c66-8716-644ba5eebcae" path="/var/lib/kubelet/pods/e57b6232-50cb-4c66-8716-644ba5eebcae/volumes"
Jul 12 00:10:18.262557 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:18.262508    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"]
Jul 12 00:10:18.262985 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:18.262764    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r" podUID="0c791880-8074-41a0-8f84-24fa41f2b0c6" containerName="istio-proxy" containerID="cri-o://f1756e53c5f278d0c634e0f9425cb3c18cdff06cff2b3db356455b3553b4bb0d" gracePeriod=30
Jul 12 00:10:19.142287 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.142251    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"]
Jul 12 00:10:19.142763 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.142740    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e57b6232-50cb-4c66-8716-644ba5eebcae" containerName="istio-proxy"
Jul 12 00:10:19.142922 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.142903    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e57b6232-50cb-4c66-8716-644ba5eebcae" containerName="istio-proxy"
Jul 12 00:10:19.143018 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.142944    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac2bf81d-1eec-4a10-bfa1-3088a64e15a0" containerName="revoke-keys"
Jul 12 00:10:19.143018 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.142951    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac2bf81d-1eec-4a10-bfa1-3088a64e15a0" containerName="revoke-keys"
Jul 12 00:10:19.143105 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.143027    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac2bf81d-1eec-4a10-bfa1-3088a64e15a0" containerName="revoke-keys"
Jul 12 00:10:19.143105 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.143036    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e57b6232-50cb-4c66-8716-644ba5eebcae" containerName="istio-proxy"
Jul 12 00:10:19.143105 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.143042    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e57b6232-50cb-4c66-8716-644ba5eebcae" containerName="istio-proxy"
Jul 12 00:10:19.146163 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.146141    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.148804 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.148774    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a7d1ef-openshift-default-dockercfg-knv4l\""
Jul 12 00:10:19.159436 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.159390    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"]
Jul 12 00:10:19.270266 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.270223    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-data\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.270266 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.270266    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-token\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.270790 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.270290    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-envoy\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.270790 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.270375    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-workload-certs\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.270790 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.270487    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-workload-socket\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.270790 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.270524    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-podinfo\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.270790 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.270544    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbsrb\" (UniqueName: \"kubernetes.io/projected/812118ce-0aed-4a5f-a3f4-681daeb32e74-kube-api-access-mbsrb\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.270790 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.270587    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-credential-socket\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.270790 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.270623    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/812118ce-0aed-4a5f-a3f4-681daeb32e74-istiod-ca-cert\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.371270 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.371232    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-workload-socket\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.371463 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.371278    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-podinfo\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.371463 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.371304    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbsrb\" (UniqueName: \"kubernetes.io/projected/812118ce-0aed-4a5f-a3f4-681daeb32e74-kube-api-access-mbsrb\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.371463 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.371339    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-credential-socket\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.371463 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.371359    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/812118ce-0aed-4a5f-a3f4-681daeb32e74-istiod-ca-cert\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.371463 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.371399    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-data\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.371463 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.371456    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-token\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.371816 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.371485    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-envoy\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.371816 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.371517    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-workload-certs\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.371919 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.371825    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-credential-socket\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.371919 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.371846    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-workload-certs\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.372025 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.371968    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-data\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.372025 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.372004    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-workload-socket\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.372233 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.372213    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/812118ce-0aed-4a5f-a3f4-681daeb32e74-istiod-ca-cert\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.373768 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.373734    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-envoy\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.374209 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.374161    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-podinfo\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.383088 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.383067    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-token\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.383190 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.383169    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbsrb\" (UniqueName: \"kubernetes.io/projected/812118ce-0aed-4a5f-a3f4-681daeb32e74-kube-api-access-mbsrb\") pod \"e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") " pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.465354 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.465261    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:19.605688 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.605655    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"]
Jul 12 00:10:19.607413 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:10:19.607376    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod812118ce_0aed_4a5f_a3f4_681daeb32e74.slice/crio-27f361d693676855e8c8721b27c619ce09db0b0f13586fa35085ab0555a376ba WatchSource:0}: Error finding container 27f361d693676855e8c8721b27c619ce09db0b0f13586fa35085ab0555a376ba: Status 404 returned error can't find the container with id 27f361d693676855e8c8721b27c619ce09db0b0f13586fa35085ab0555a376ba
Jul 12 00:10:19.609584 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.609550    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:10:19.609675 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.609619    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:10:19.609675 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.609650    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:10:19.952022 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.951985    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd" event={"ID":"812118ce-0aed-4a5f-a3f4-681daeb32e74","Type":"ContainerStarted","Data":"af3d1a4cd36df93cd66f667b6d479d41caff5276ba1ffc4893d8bf05ad7ab484"}
Jul 12 00:10:19.952177 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.952050    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd" event={"ID":"812118ce-0aed-4a5f-a3f4-681daeb32e74","Type":"ContainerStarted","Data":"27f361d693676855e8c8721b27c619ce09db0b0f13586fa35085ab0555a376ba"}
Jul 12 00:10:19.973496 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:19.973436    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd" podStartSLOduration=0.97340161 podStartE2EDuration="973.40161ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:10:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:10:19.970856142 +0000 UTC m=+2349.548948919" watchObservedRunningTime="2026-07-12 00:10:19.97340161 +0000 UTC m=+2349.551494389"
Jul 12 00:10:20.189909 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:20.189874    2569 patch_prober.go:28] interesting pod/e2e-api-b-a58284-openshift-default-57667b9977-fk65r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 12 00:10:20.190050 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:20.189933    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r" podUID="0c791880-8074-41a0-8f84-24fa41f2b0c6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 12 00:10:20.465819 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:20.465779    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:20.467412 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:20.467387    2569 patch_prober.go:28] interesting pod/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 12 00:10:20.467564 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:20.467455    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd" podUID="812118ce-0aed-4a5f-a3f4-681daeb32e74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 12 00:10:21.470026 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:21.469996    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:21.963211 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:21.963179    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:21.964343 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:21.964320    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:23.522049 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.522019    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:10:23.613239 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.613213    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-698q5\" (UniqueName: \"kubernetes.io/projected/0c791880-8074-41a0-8f84-24fa41f2b0c6-kube-api-access-698q5\") pod \"0c791880-8074-41a0-8f84-24fa41f2b0c6\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") "
Jul 12 00:10:23.613446 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.613246    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-workload-socket\") pod \"0c791880-8074-41a0-8f84-24fa41f2b0c6\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") "
Jul 12 00:10:23.613446 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.613279    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c791880-8074-41a0-8f84-24fa41f2b0c6-istiod-ca-cert\") pod \"0c791880-8074-41a0-8f84-24fa41f2b0c6\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") "
Jul 12 00:10:23.613446 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.613296    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-envoy\") pod \"0c791880-8074-41a0-8f84-24fa41f2b0c6\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") "
Jul 12 00:10:23.613639 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.613447    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-data\") pod \"0c791880-8074-41a0-8f84-24fa41f2b0c6\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") "
Jul 12 00:10:23.613639 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.613501    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-credential-socket\") pod \"0c791880-8074-41a0-8f84-24fa41f2b0c6\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") "
Jul 12 00:10:23.613639 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.613528    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-podinfo\") pod \"0c791880-8074-41a0-8f84-24fa41f2b0c6\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") "
Jul 12 00:10:23.613639 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.613555    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-workload-certs\") pod \"0c791880-8074-41a0-8f84-24fa41f2b0c6\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") "
Jul 12 00:10:23.613639 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.613594    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-token\") pod \"0c791880-8074-41a0-8f84-24fa41f2b0c6\" (UID: \"0c791880-8074-41a0-8f84-24fa41f2b0c6\") "
Jul 12 00:10:23.613639 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.613614    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0c791880-8074-41a0-8f84-24fa41f2b0c6" (UID: "0c791880-8074-41a0-8f84-24fa41f2b0c6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:10:23.613933 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.613753    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0c791880-8074-41a0-8f84-24fa41f2b0c6" (UID: "0c791880-8074-41a0-8f84-24fa41f2b0c6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:10:23.613933 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.613776    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c791880-8074-41a0-8f84-24fa41f2b0c6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0c791880-8074-41a0-8f84-24fa41f2b0c6" (UID: "0c791880-8074-41a0-8f84-24fa41f2b0c6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:10:23.613933 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.613855    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0c791880-8074-41a0-8f84-24fa41f2b0c6" (UID: "0c791880-8074-41a0-8f84-24fa41f2b0c6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:10:23.614096 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.613959    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-workload-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:10:23.614096 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.613978    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c791880-8074-41a0-8f84-24fa41f2b0c6-istiod-ca-cert\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:10:23.614096 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.613993    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-credential-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:10:23.614096 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.614008    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-workload-certs\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:10:23.614304 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.614227    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-data" (OuterVolumeSpecName: "istio-data") pod "0c791880-8074-41a0-8f84-24fa41f2b0c6" (UID: "0c791880-8074-41a0-8f84-24fa41f2b0c6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:10:23.615628 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.615606    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0c791880-8074-41a0-8f84-24fa41f2b0c6" (UID: "0c791880-8074-41a0-8f84-24fa41f2b0c6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:10:23.615746 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.615712    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-token" (OuterVolumeSpecName: "istio-token") pod "0c791880-8074-41a0-8f84-24fa41f2b0c6" (UID: "0c791880-8074-41a0-8f84-24fa41f2b0c6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:10:23.615746 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.615730    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c791880-8074-41a0-8f84-24fa41f2b0c6-kube-api-access-698q5" (OuterVolumeSpecName: "kube-api-access-698q5") pod "0c791880-8074-41a0-8f84-24fa41f2b0c6" (UID: "0c791880-8074-41a0-8f84-24fa41f2b0c6"). InnerVolumeSpecName "kube-api-access-698q5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:10:23.615830 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.615758    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0c791880-8074-41a0-8f84-24fa41f2b0c6" (UID: "0c791880-8074-41a0-8f84-24fa41f2b0c6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 12 00:10:23.714842 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.714812    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-envoy\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:10:23.714842 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.714840    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-data\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:10:23.715026 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.714852    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-podinfo\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:10:23.715026 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.714865    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c791880-8074-41a0-8f84-24fa41f2b0c6-istio-token\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:10:23.715026 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.714877    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-698q5\" (UniqueName: \"kubernetes.io/projected/0c791880-8074-41a0-8f84-24fa41f2b0c6-kube-api-access-698q5\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:10:23.975515 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.975407    2569 generic.go:356] "Generic (PLEG): container finished" podID="0c791880-8074-41a0-8f84-24fa41f2b0c6" containerID="f1756e53c5f278d0c634e0f9425cb3c18cdff06cff2b3db356455b3553b4bb0d" exitCode=0
Jul 12 00:10:23.975515 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.975487    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"
Jul 12 00:10:23.975515 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.975499    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r" event={"ID":"0c791880-8074-41a0-8f84-24fa41f2b0c6","Type":"ContainerDied","Data":"f1756e53c5f278d0c634e0f9425cb3c18cdff06cff2b3db356455b3553b4bb0d"}
Jul 12 00:10:23.975727 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.975591    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r" event={"ID":"0c791880-8074-41a0-8f84-24fa41f2b0c6","Type":"ContainerDied","Data":"cb784443e9131b3e5f6cb8f464c5ef74779127ada6f05522915bb4ae0e156a61"}
Jul 12 00:10:23.975727 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.975615    2569 scope.go:117] "RemoveContainer" containerID="f1756e53c5f278d0c634e0f9425cb3c18cdff06cff2b3db356455b3553b4bb0d"
Jul 12 00:10:23.985821 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.985803    2569 scope.go:117] "RemoveContainer" containerID="1abf10c214ae00fab0a8ee784c1fc3991784464fa0d10865743a43d1ba2affa1"
Jul 12 00:10:23.993941 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.993924    2569 scope.go:117] "RemoveContainer" containerID="f1756e53c5f278d0c634e0f9425cb3c18cdff06cff2b3db356455b3553b4bb0d"
Jul 12 00:10:23.994256 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:10:23.994235    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1756e53c5f278d0c634e0f9425cb3c18cdff06cff2b3db356455b3553b4bb0d\": container with ID starting with f1756e53c5f278d0c634e0f9425cb3c18cdff06cff2b3db356455b3553b4bb0d not found: ID does not exist" containerID="f1756e53c5f278d0c634e0f9425cb3c18cdff06cff2b3db356455b3553b4bb0d"
Jul 12 00:10:23.994321 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.994266    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1756e53c5f278d0c634e0f9425cb3c18cdff06cff2b3db356455b3553b4bb0d"} err="failed to get container status \"f1756e53c5f278d0c634e0f9425cb3c18cdff06cff2b3db356455b3553b4bb0d\": rpc error: code = NotFound desc = could not find container \"f1756e53c5f278d0c634e0f9425cb3c18cdff06cff2b3db356455b3553b4bb0d\": container with ID starting with f1756e53c5f278d0c634e0f9425cb3c18cdff06cff2b3db356455b3553b4bb0d not found: ID does not exist"
Jul 12 00:10:23.994321 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.994285    2569 scope.go:117] "RemoveContainer" containerID="1abf10c214ae00fab0a8ee784c1fc3991784464fa0d10865743a43d1ba2affa1"
Jul 12 00:10:23.994598 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:10:23.994572    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1abf10c214ae00fab0a8ee784c1fc3991784464fa0d10865743a43d1ba2affa1\": container with ID starting with 1abf10c214ae00fab0a8ee784c1fc3991784464fa0d10865743a43d1ba2affa1 not found: ID does not exist" containerID="1abf10c214ae00fab0a8ee784c1fc3991784464fa0d10865743a43d1ba2affa1"
Jul 12 00:10:23.994978 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.994604    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1abf10c214ae00fab0a8ee784c1fc3991784464fa0d10865743a43d1ba2affa1"} err="failed to get container status \"1abf10c214ae00fab0a8ee784c1fc3991784464fa0d10865743a43d1ba2affa1\": rpc error: code = NotFound desc = could not find container \"1abf10c214ae00fab0a8ee784c1fc3991784464fa0d10865743a43d1ba2affa1\": container with ID starting with 1abf10c214ae00fab0a8ee784c1fc3991784464fa0d10865743a43d1ba2affa1 not found: ID does not exist"
Jul 12 00:10:23.997460 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:23.997435    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"]
Jul 12 00:10:24.000570 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:24.000550    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a58284-openshift-default-57667b9977-fk65r"]
Jul 12 00:10:24.983075 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:24.983042    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c791880-8074-41a0-8f84-24fa41f2b0c6" path="/var/lib/kubelet/pods/0c791880-8074-41a0-8f84-24fa41f2b0c6/volumes"
Jul 12 00:10:27.053302 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.053265    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"]
Jul 12 00:10:27.053784 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.053688    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c791880-8074-41a0-8f84-24fa41f2b0c6" containerName="istio-proxy"
Jul 12 00:10:27.053784 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.053704    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c791880-8074-41a0-8f84-24fa41f2b0c6" containerName="istio-proxy"
Jul 12 00:10:27.053784 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.053717    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e57b6232-50cb-4c66-8716-644ba5eebcae" containerName="istio-proxy"
Jul 12 00:10:27.053784 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.053723    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e57b6232-50cb-4c66-8716-644ba5eebcae" containerName="istio-proxy"
Jul 12 00:10:27.053784 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.053731    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c791880-8074-41a0-8f84-24fa41f2b0c6" containerName="istio-proxy"
Jul 12 00:10:27.053784 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.053737    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c791880-8074-41a0-8f84-24fa41f2b0c6" containerName="istio-proxy"
Jul 12 00:10:27.054102 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.053802    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c791880-8074-41a0-8f84-24fa41f2b0c6" containerName="istio-proxy"
Jul 12 00:10:27.054102 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.053812    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c791880-8074-41a0-8f84-24fa41f2b0c6" containerName="istio-proxy"
Jul 12 00:10:27.058267 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.058246    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.060844 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.060819    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-a2ea47-openshift-default-dockercfg-stkhg\""
Jul 12 00:10:27.066438 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.066400    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"]
Jul 12 00:10:27.144333 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.144292    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-token\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.144558 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.144438    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-workload-socket\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.144558 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.144478    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-data\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.144558 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.144526    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-podinfo\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.144762 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.144566    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-workload-certs\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.144762 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.144591    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t85z2\" (UniqueName: \"kubernetes.io/projected/274302f9-dac6-4ac0-9e22-0c12651ee624-kube-api-access-t85z2\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.144879 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.144805    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-envoy\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.144935 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.144876    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-credential-socket\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.144935 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.144912    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/274302f9-dac6-4ac0-9e22-0c12651ee624-istiod-ca-cert\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.245599 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.245561    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-token\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.245787 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.245630    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-workload-socket\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.245787 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.245656    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-data\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.245787 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.245692    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-podinfo\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.245787 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.245729    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-workload-certs\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.245787 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.245750    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t85z2\" (UniqueName: \"kubernetes.io/projected/274302f9-dac6-4ac0-9e22-0c12651ee624-kube-api-access-t85z2\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.246104 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.245800    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-envoy\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.246104 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.245847    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-credential-socket\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.246104 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.245878    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/274302f9-dac6-4ac0-9e22-0c12651ee624-istiod-ca-cert\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.246325 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.246294    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-credential-socket\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.246492 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.246368    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-workload-socket\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.246492 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.246461    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-workload-certs\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.246611 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.246516    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-data\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.246611 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.246580    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/274302f9-dac6-4ac0-9e22-0c12651ee624-istiod-ca-cert\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.248489 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.248462    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-envoy\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.248693 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.248679    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-podinfo\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.253043 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.253019    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-token\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.253186 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.253160    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t85z2\" (UniqueName: \"kubernetes.io/projected/274302f9-dac6-4ac0-9e22-0c12651ee624-kube-api-access-t85z2\") pod \"e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") " pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.373009 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.372917    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:27.512940 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.512904    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"]
Jul 12 00:10:27.514131 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:10:27.514102    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod274302f9_dac6_4ac0_9e22_0c12651ee624.slice/crio-b14be02b1035133c8f8ff19610f9f98effca91c818ba00eba2ecc00991f965b1 WatchSource:0}: Error finding container b14be02b1035133c8f8ff19610f9f98effca91c818ba00eba2ecc00991f965b1: Status 404 returned error can't find the container with id b14be02b1035133c8f8ff19610f9f98effca91c818ba00eba2ecc00991f965b1
Jul 12 00:10:27.516186 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.516153    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:10:27.516284 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.516230    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:10:27.516284 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:27.516275    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:10:28.004295 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:28.004258    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk" event={"ID":"274302f9-dac6-4ac0-9e22-0c12651ee624","Type":"ContainerStarted","Data":"3f5c440b315105854f2286f2382448bcf684135d4c131a08633b2c64cffaa4e9"}
Jul 12 00:10:28.004527 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:28.004332    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk" event={"ID":"274302f9-dac6-4ac0-9e22-0c12651ee624","Type":"ContainerStarted","Data":"b14be02b1035133c8f8ff19610f9f98effca91c818ba00eba2ecc00991f965b1"}
Jul 12 00:10:28.026909 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:28.026848    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk" podStartSLOduration=1.02683411 podStartE2EDuration="1.02683411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:10:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:10:28.023823933 +0000 UTC m=+2357.601916735" watchObservedRunningTime="2026-07-12 00:10:28.02683411 +0000 UTC m=+2357.604926885"
Jul 12 00:10:28.375392 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:28.375353    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:28.383217 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:28.383189    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:29.009267 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:29.009229    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:29.010311 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:29.010290    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:36.046561 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:36.046525    2569 generic.go:356] "Generic (PLEG): container finished" podID="812118ce-0aed-4a5f-a3f4-681daeb32e74" containerID="af3d1a4cd36df93cd66f667b6d479d41caff5276ba1ffc4893d8bf05ad7ab484" exitCode=0
Jul 12 00:10:36.047101 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:36.046603    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd" event={"ID":"812118ce-0aed-4a5f-a3f4-681daeb32e74","Type":"ContainerDied","Data":"af3d1a4cd36df93cd66f667b6d479d41caff5276ba1ffc4893d8bf05ad7ab484"}
Jul 12 00:10:36.047178 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:36.047120    2569 scope.go:117] "RemoveContainer" containerID="af3d1a4cd36df93cd66f667b6d479d41caff5276ba1ffc4893d8bf05ad7ab484"
Jul 12 00:10:36.048139 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:36.048109    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:10:36.048242 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:36.048176    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:10:36.048242 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:36.048207    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:10:36.963584 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:36.963546    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:37.059329 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:37.059289    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd" event={"ID":"812118ce-0aed-4a5f-a3f4-681daeb32e74","Type":"ContainerStarted","Data":"5517e0ab17d07334a18b40e3778d8ba7111b2843fc8cf226ef0b795b09a7b7c1"}
Jul 12 00:10:37.468049 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:37.468002    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:37.468613 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:37.468195    2569 patch_prober.go:28] interesting pod/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 12 00:10:37.468871 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:37.468825    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd" podUID="812118ce-0aed-4a5f-a3f4-681daeb32e74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 12 00:10:38.466723 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:38.466684    2569 patch_prober.go:28] interesting pod/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 12 00:10:38.467339 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:38.467300    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd" podUID="812118ce-0aed-4a5f-a3f4-681daeb32e74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 12 00:10:39.466650 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:39.466602    2569 patch_prober.go:28] interesting pod/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 12 00:10:39.466883 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:39.466674    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd" podUID="812118ce-0aed-4a5f-a3f4-681daeb32e74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 12 00:10:40.466310 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:40.465838    2569 patch_prober.go:28] interesting pod/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Jul 12 00:10:40.466310 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:40.465906    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd" podUID="812118ce-0aed-4a5f-a3f4-681daeb32e74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Jul 12 00:10:41.471541 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:41.471506    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:41.472015 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:41.471914    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:41.472663 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:41.472643    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:10:44.014299 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:44.013529    2569 patch_prober.go:28] interesting pod/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": read tcp 10.133.0.2:51104->10.133.0.68:15021: read: connection reset by peer" start-of-body=
Jul 12 00:10:44.014299 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:44.013595    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk" podUID="274302f9-dac6-4ac0-9e22-0c12651ee624" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": read tcp 10.133.0.2:51104->10.133.0.68:15021: read: connection reset by peer"
Jul 12 00:10:44.108183 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:44.108149    2569 generic.go:356] "Generic (PLEG): container finished" podID="274302f9-dac6-4ac0-9e22-0c12651ee624" containerID="3f5c440b315105854f2286f2382448bcf684135d4c131a08633b2c64cffaa4e9" exitCode=0
Jul 12 00:10:44.108343 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:44.108223    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk" event={"ID":"274302f9-dac6-4ac0-9e22-0c12651ee624","Type":"ContainerDied","Data":"3f5c440b315105854f2286f2382448bcf684135d4c131a08633b2c64cffaa4e9"}
Jul 12 00:10:44.108641 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:44.108624    2569 scope.go:117] "RemoveContainer" containerID="3f5c440b315105854f2286f2382448bcf684135d4c131a08633b2c64cffaa4e9"
Jul 12 00:10:44.109629 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:44.109600    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:10:44.109723 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:44.109673    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:10:44.109767 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:44.109729    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 12 00:10:45.114682 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:45.114650    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk" event={"ID":"274302f9-dac6-4ac0-9e22-0c12651ee624","Type":"ContainerStarted","Data":"7ae0dd036fdadf6e14b219dbef45ab32b16e90cda1982c21201eb93c80638312"}
Jul 12 00:10:45.373859 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:45.373755    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:45.374049 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:45.373890    2569 patch_prober.go:28] interesting pod/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Jul 12 00:10:45.374049 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:45.373934    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk" podUID="274302f9-dac6-4ac0-9e22-0c12651ee624" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Jul 12 00:10:46.373983 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:46.373941    2569 patch_prober.go:28] interesting pod/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Jul 12 00:10:46.374606 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:46.374569    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk" podUID="274302f9-dac6-4ac0-9e22-0c12651ee624" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Jul 12 00:10:47.373977 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:47.373932    2569 patch_prober.go:28] interesting pod/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Jul 12 00:10:47.374195 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:47.374006    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk" podUID="274302f9-dac6-4ac0-9e22-0c12651ee624" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Jul 12 00:10:48.374048 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:48.374008    2569 patch_prober.go:28] interesting pod/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Jul 12 00:10:48.374360 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:48.374330    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk" podUID="274302f9-dac6-4ac0-9e22-0c12651ee624" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Jul 12 00:10:49.376480 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:49.376446    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:49.376874 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:49.376705    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:10:49.377460 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:10:49.377445    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:11:11.949689 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:11:11.949657    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 12 00:11:11.953457 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:11:11.953419    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 12 00:15:00.131758 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:00.131675    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2ea47-29730255-pmn95"]
Jul 12 00:15:00.135507 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:00.135483    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7d1ef-29730255-nnpdc"]
Jul 12 00:15:00.135637 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:00.135603    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2ea47-29730255-pmn95"
Jul 12 00:15:00.138030 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:00.138005    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\""
Jul 12 00:15:00.138705 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:00.138684    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29730255-rzkk7"]
Jul 12 00:15:00.138815 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:00.138786    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7d1ef-29730255-nnpdc"
Jul 12 00:15:00.141847 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:00.141824    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730255-rzkk7"
Jul 12 00:15:00.143613 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:00.143594    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2ea47-29730255-pmn95"]
Jul 12 00:15:00.146631 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:00.146611    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7d1ef-29730255-nnpdc"]
Jul 12 00:15:00.148896 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:00.148876    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2ea47-29730255-pmn95"
Jul 12 00:15:00.158311 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:00.158285    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29730255-rzkk7"]
Jul 12 00:15:00.158520 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:00.158374    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7d1ef-29730255-nnpdc"
Jul 12 00:15:00.198211 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:00.198184    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730255-rzkk7"
Jul 12 00:15:00.314128 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:00.314079    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2ea47-29730255-pmn95"]
Jul 12 00:15:00.316525 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:15:00.316490    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bdbcfe0_684a_4190_94ef_02308e761382.slice/crio-95f2a1be04afba2346c0d0aaffb0e2661e9ac93336230e68854a4369924d9ad3 WatchSource:0}: Error finding container 95f2a1be04afba2346c0d0aaffb0e2661e9ac93336230e68854a4369924d9ad3: Status 404 returned error can't find the container with id 95f2a1be04afba2346c0d0aaffb0e2661e9ac93336230e68854a4369924d9ad3
Jul 12 00:15:00.373648 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:00.373608    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29730255-rzkk7"]
Jul 12 00:15:00.375220 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:15:00.375187    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod435e74ad_4eb4_472e_9eac_40311e97f3f2.slice/crio-6f3bec7448bbca27a32c847d5f051aab0942a36c15c63a701ed49cb30c72e66d WatchSource:0}: Error finding container 6f3bec7448bbca27a32c847d5f051aab0942a36c15c63a701ed49cb30c72e66d: Status 404 returned error can't find the container with id 6f3bec7448bbca27a32c847d5f051aab0942a36c15c63a701ed49cb30c72e66d
Jul 12 00:15:00.413588 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:00.413554    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730255-rzkk7" event={"ID":"435e74ad-4eb4-472e-9eac-40311e97f3f2","Type":"ContainerStarted","Data":"6f3bec7448bbca27a32c847d5f051aab0942a36c15c63a701ed49cb30c72e66d"}
Jul 12 00:15:00.414740 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:00.414716    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2ea47-29730255-pmn95" event={"ID":"9bdbcfe0-684a-4190-94ef-02308e761382","Type":"ContainerStarted","Data":"95f2a1be04afba2346c0d0aaffb0e2661e9ac93336230e68854a4369924d9ad3"}
Jul 12 00:15:00.549068 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:00.549038    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7d1ef-29730255-nnpdc"]
Jul 12 00:15:00.550538 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:15:00.550505    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod895acadf_58cc_4a6a_b001_db362d1a6979.slice/crio-64b688c2ae23db2e21c4d3c654117043cbc1f1292465a1843950993ed5269764 WatchSource:0}: Error finding container 64b688c2ae23db2e21c4d3c654117043cbc1f1292465a1843950993ed5269764: Status 404 returned error can't find the container with id 64b688c2ae23db2e21c4d3c654117043cbc1f1292465a1843950993ed5269764
Jul 12 00:15:01.421062 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:01.421019    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730255-rzkk7" event={"ID":"435e74ad-4eb4-472e-9eac-40311e97f3f2","Type":"ContainerStarted","Data":"4d7b8415fd2878625da550fe37376a0b6667b685e75dc0438a10383e0c8470c9"}
Jul 12 00:15:01.422450 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:01.422407    2569 generic.go:356] "Generic (PLEG): container finished" podID="9bdbcfe0-684a-4190-94ef-02308e761382" containerID="34ff43a453eca2a1e2ac267d07fcc30260f3ed89cbe3d754f4fa657940933a79" exitCode=0
Jul 12 00:15:01.422564 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:01.422496    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2ea47-29730255-pmn95" event={"ID":"9bdbcfe0-684a-4190-94ef-02308e761382","Type":"ContainerDied","Data":"34ff43a453eca2a1e2ac267d07fcc30260f3ed89cbe3d754f4fa657940933a79"}
Jul 12 00:15:01.423945 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:01.423926    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7d1ef-29730255-nnpdc" event={"ID":"895acadf-58cc-4a6a-b001-db362d1a6979","Type":"ContainerStarted","Data":"03721aa511145d896a335054b6cd321aaa593484283759ce6d0fce6815a60f6f"}
Jul 12 00:15:01.424017 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:01.423966    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7d1ef-29730255-nnpdc" event={"ID":"895acadf-58cc-4a6a-b001-db362d1a6979","Type":"ContainerStarted","Data":"64b688c2ae23db2e21c4d3c654117043cbc1f1292465a1843950993ed5269764"}
Jul 12 00:15:01.436470 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:01.436410    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730255-rzkk7" podStartSLOduration=1.436394699 podStartE2EDuration="1.436394699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:15:01.434855903 +0000 UTC m=+2631.012948680" watchObservedRunningTime="2026-07-12 00:15:01.436394699 +0000 UTC m=+2631.014487504"
Jul 12 00:15:01.462732 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:01.462672    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7d1ef-29730255-nnpdc" podStartSLOduration=1.462657555 podStartE2EDuration="1.462657555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:15:01.460265462 +0000 UTC m=+2631.038358240" watchObservedRunningTime="2026-07-12 00:15:01.462657555 +0000 UTC m=+2631.040750330"
Jul 12 00:15:02.430078 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:02.430042    2569 generic.go:356] "Generic (PLEG): container finished" podID="435e74ad-4eb4-472e-9eac-40311e97f3f2" containerID="4d7b8415fd2878625da550fe37376a0b6667b685e75dc0438a10383e0c8470c9" exitCode=0
Jul 12 00:15:02.430518 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:02.430131    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730255-rzkk7" event={"ID":"435e74ad-4eb4-472e-9eac-40311e97f3f2","Type":"ContainerDied","Data":"4d7b8415fd2878625da550fe37376a0b6667b685e75dc0438a10383e0c8470c9"}
Jul 12 00:15:02.431499 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:02.431478    2569 generic.go:356] "Generic (PLEG): container finished" podID="895acadf-58cc-4a6a-b001-db362d1a6979" containerID="03721aa511145d896a335054b6cd321aaa593484283759ce6d0fce6815a60f6f" exitCode=0
Jul 12 00:15:02.431604 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:02.431558    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7d1ef-29730255-nnpdc" event={"ID":"895acadf-58cc-4a6a-b001-db362d1a6979","Type":"ContainerDied","Data":"03721aa511145d896a335054b6cd321aaa593484283759ce6d0fce6815a60f6f"}
Jul 12 00:15:02.570454 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:02.570414    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2ea47-29730255-pmn95"
Jul 12 00:15:03.438549 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:03.438521    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2ea47-29730255-pmn95"
Jul 12 00:15:03.438975 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:03.438564    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2ea47-29730255-pmn95" event={"ID":"9bdbcfe0-684a-4190-94ef-02308e761382","Type":"ContainerDied","Data":"95f2a1be04afba2346c0d0aaffb0e2661e9ac93336230e68854a4369924d9ad3"}
Jul 12 00:15:03.438975 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:03.438618    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="95f2a1be04afba2346c0d0aaffb0e2661e9ac93336230e68854a4369924d9ad3"
Jul 12 00:15:03.577689 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:03.577663    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730255-rzkk7"
Jul 12 00:15:03.604096 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:03.604069    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7d1ef-29730255-nnpdc"
Jul 12 00:15:04.445878 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:04.445853    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730255-rzkk7"
Jul 12 00:15:04.446295 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:04.445900    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29730255-rzkk7" event={"ID":"435e74ad-4eb4-472e-9eac-40311e97f3f2","Type":"ContainerDied","Data":"6f3bec7448bbca27a32c847d5f051aab0942a36c15c63a701ed49cb30c72e66d"}
Jul 12 00:15:04.446295 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:04.445950    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f3bec7448bbca27a32c847d5f051aab0942a36c15c63a701ed49cb30c72e66d"
Jul 12 00:15:04.447165 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:04.447145    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7d1ef-29730255-nnpdc"
Jul 12 00:15:04.447276 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:04.447146    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7d1ef-29730255-nnpdc" event={"ID":"895acadf-58cc-4a6a-b001-db362d1a6979","Type":"ContainerDied","Data":"64b688c2ae23db2e21c4d3c654117043cbc1f1292465a1843950993ed5269764"}
Jul 12 00:15:04.447276 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:15:04.447256    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64b688c2ae23db2e21c4d3c654117043cbc1f1292465a1843950993ed5269764"
Jul 12 00:16:11.980740 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:11.980663    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 12 00:16:11.984548 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:11.984530    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xd2gc_9f1aaf88-8ea0-4105-b62b-f784d91165d1/ovn-acl-logging/0.log"
Jul 12 00:16:32.983970 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:32.983931    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr"]
Jul 12 00:16:32.984617 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:32.984472    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bdbcfe0-684a-4190-94ef-02308e761382" containerName="cleanup"
Jul 12 00:16:32.984617 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:32.984493    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bdbcfe0-684a-4190-94ef-02308e761382" containerName="cleanup"
Jul 12 00:16:32.984617 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:32.984505    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="435e74ad-4eb4-472e-9eac-40311e97f3f2" containerName="cleanup"
Jul 12 00:16:32.984617 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:32.984514    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="435e74ad-4eb4-472e-9eac-40311e97f3f2" containerName="cleanup"
Jul 12 00:16:32.984617 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:32.984524    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="895acadf-58cc-4a6a-b001-db362d1a6979" containerName="cleanup"
Jul 12 00:16:32.984617 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:32.984532    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="895acadf-58cc-4a6a-b001-db362d1a6979" containerName="cleanup"
Jul 12 00:16:32.984962 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:32.984635    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="895acadf-58cc-4a6a-b001-db362d1a6979" containerName="cleanup"
Jul 12 00:16:32.984962 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:32.984647    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bdbcfe0-684a-4190-94ef-02308e761382" containerName="cleanup"
Jul 12 00:16:32.984962 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:32.984657    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="435e74ad-4eb4-472e-9eac-40311e97f3f2" containerName="cleanup"
Jul 12 00:16:32.988030 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:32.987995    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr"]
Jul 12 00:16:32.988180 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:32.988116    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr"
Jul 12 00:16:32.990655 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:32.990624    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\""
Jul 12 00:16:33.117677 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:33.117638    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/433ebee3-1422-47f5-acd4-2b0943f4d526-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr\" (UID: \"433ebee3-1422-47f5-acd4-2b0943f4d526\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr"
Jul 12 00:16:33.218684 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:33.218652    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/433ebee3-1422-47f5-acd4-2b0943f4d526-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr\" (UID: \"433ebee3-1422-47f5-acd4-2b0943f4d526\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr"
Jul 12 00:16:33.219269 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:33.219251    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/433ebee3-1422-47f5-acd4-2b0943f4d526-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr\" (UID: \"433ebee3-1422-47f5-acd4-2b0943f4d526\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr"
Jul 12 00:16:33.298800 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:33.298765    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr"
Jul 12 00:16:33.435918 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:33.435889    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr"]
Jul 12 00:16:33.438327 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:16:33.438297    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod433ebee3_1422_47f5_acd4_2b0943f4d526.slice/crio-bc42225423bc3550b5303e62273e0c831720c64f7706937ac7b39e1c0c3d594e WatchSource:0}: Error finding container bc42225423bc3550b5303e62273e0c831720c64f7706937ac7b39e1c0c3d594e: Status 404 returned error can't find the container with id bc42225423bc3550b5303e62273e0c831720c64f7706937ac7b39e1c0c3d594e
Jul 12 00:16:33.915160 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:33.915115    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr" event={"ID":"433ebee3-1422-47f5-acd4-2b0943f4d526","Type":"ContainerStarted","Data":"d15a22b4e1d2cec5ed39f1825032f1e8791602b1dd47e70741ffb775d81e3edf"}
Jul 12 00:16:33.915160 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:33.915168    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr" event={"ID":"433ebee3-1422-47f5-acd4-2b0943f4d526","Type":"ContainerStarted","Data":"bc42225423bc3550b5303e62273e0c831720c64f7706937ac7b39e1c0c3d594e"}
Jul 12 00:16:33.931225 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:33.931130    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr" podStartSLOduration=1.931115024 podStartE2EDuration="1.931115024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:16:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:16:33.928625892 +0000 UTC m=+2723.506718669" watchObservedRunningTime="2026-07-12 00:16:33.931115024 +0000 UTC m=+2723.509207800"
Jul 12 00:16:34.922591 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:34.922550    2569 generic.go:356] "Generic (PLEG): container finished" podID="433ebee3-1422-47f5-acd4-2b0943f4d526" containerID="d15a22b4e1d2cec5ed39f1825032f1e8791602b1dd47e70741ffb775d81e3edf" exitCode=0
Jul 12 00:16:34.922972 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:34.922635    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr" event={"ID":"433ebee3-1422-47f5-acd4-2b0943f4d526","Type":"ContainerDied","Data":"d15a22b4e1d2cec5ed39f1825032f1e8791602b1dd47e70741ffb775d81e3edf"}
Jul 12 00:16:36.058992 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:36.058963    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr"
Jul 12 00:16:36.147810 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:36.147775    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/433ebee3-1422-47f5-acd4-2b0943f4d526-maas-api-service-ca\") pod \"433ebee3-1422-47f5-acd4-2b0943f4d526\" (UID: \"433ebee3-1422-47f5-acd4-2b0943f4d526\") "
Jul 12 00:16:36.148195 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:36.148175    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/433ebee3-1422-47f5-acd4-2b0943f4d526-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "433ebee3-1422-47f5-acd4-2b0943f4d526" (UID: "433ebee3-1422-47f5-acd4-2b0943f4d526"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:16:36.249333 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:36.249244    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/433ebee3-1422-47f5-acd4-2b0943f4d526-maas-api-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:16:36.934137 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:36.934103    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr"
Jul 12 00:16:36.934318 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:36.934141    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr" event={"ID":"433ebee3-1422-47f5-acd4-2b0943f4d526","Type":"ContainerDied","Data":"bc42225423bc3550b5303e62273e0c831720c64f7706937ac7b39e1c0c3d594e"}
Jul 12 00:16:36.934318 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:36.934215    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc42225423bc3550b5303e62273e0c831720c64f7706937ac7b39e1c0c3d594e"
Jul 12 00:16:43.362457 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:43.362406    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7d1ef-29730255-nnpdc"]
Jul 12 00:16:43.366217 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:43.366184    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-a7d1ef-29730255-nnpdc"]
Jul 12 00:16:44.982092 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:44.982059    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="895acadf-58cc-4a6a-b001-db362d1a6979" path="/var/lib/kubelet/pods/895acadf-58cc-4a6a-b001-db362d1a6979/volumes"
Jul 12 00:16:58.704350 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:58.704316    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"]
Jul 12 00:16:58.704940 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:58.704549    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd" podUID="812118ce-0aed-4a5f-a3f4-681daeb32e74" containerName="istio-proxy" containerID="cri-o://5517e0ab17d07334a18b40e3778d8ba7111b2843fc8cf226ef0b795b09a7b7c1" gracePeriod=30
Jul 12 00:16:59.070818 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:59.070779    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g"]
Jul 12 00:16:59.071182 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:59.071169    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="433ebee3-1422-47f5-acd4-2b0943f4d526" containerName="revoke-keys"
Jul 12 00:16:59.071241 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:59.071183    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="433ebee3-1422-47f5-acd4-2b0943f4d526" containerName="revoke-keys"
Jul 12 00:16:59.071281 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:59.071247    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="433ebee3-1422-47f5-acd4-2b0943f4d526" containerName="revoke-keys"
Jul 12 00:16:59.074389 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:59.074372    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g"
Jul 12 00:16:59.077032 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:59.077012    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\""
Jul 12 00:16:59.082028 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:59.082005    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g"]
Jul 12 00:16:59.153958 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:59.153921    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1547a8e7-59c0-4ad6-be99-f595b4c3d1cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g\" (UID: \"1547a8e7-59c0-4ad6-be99-f595b4c3d1cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g"
Jul 12 00:16:59.255322 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:59.255274    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1547a8e7-59c0-4ad6-be99-f595b4c3d1cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g\" (UID: \"1547a8e7-59c0-4ad6-be99-f595b4c3d1cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g"
Jul 12 00:16:59.255947 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:59.255927    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1547a8e7-59c0-4ad6-be99-f595b4c3d1cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g\" (UID: \"1547a8e7-59c0-4ad6-be99-f595b4c3d1cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g"
Jul 12 00:16:59.386486 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:59.386381    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g"
Jul 12 00:16:59.524558 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:16:59.524530    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g"]
Jul 12 00:16:59.526375 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:16:59.526343    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1547a8e7_59c0_4ad6_be99_f595b4c3d1cc.slice/crio-343a87d1c42bf4278d792fbe81c3027eec03869fefe20da395b206eac358d425 WatchSource:0}: Error finding container 343a87d1c42bf4278d792fbe81c3027eec03869fefe20da395b206eac358d425: Status 404 returned error can't find the container with id 343a87d1c42bf4278d792fbe81c3027eec03869fefe20da395b206eac358d425
Jul 12 00:17:00.051639 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:00.051607    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g" event={"ID":"1547a8e7-59c0-4ad6-be99-f595b4c3d1cc","Type":"ContainerStarted","Data":"290245adf244f682bcdf02ee1a51b9fdcbc62635366badfdfdcd1e29dfeacfda"}
Jul 12 00:17:00.052042 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:00.051665    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g" event={"ID":"1547a8e7-59c0-4ad6-be99-f595b4c3d1cc","Type":"ContainerStarted","Data":"343a87d1c42bf4278d792fbe81c3027eec03869fefe20da395b206eac358d425"}
Jul 12 00:17:00.069818 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:00.069769    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g" podStartSLOduration=1.069752905 podStartE2EDuration="1.069752905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:16:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:17:00.067147528 +0000 UTC m=+2749.645240324" watchObservedRunningTime="2026-07-12 00:17:00.069752905 +0000 UTC m=+2749.647845681"
Jul 12 00:17:01.058006 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:01.057970    2569 generic.go:356] "Generic (PLEG): container finished" podID="1547a8e7-59c0-4ad6-be99-f595b4c3d1cc" containerID="290245adf244f682bcdf02ee1a51b9fdcbc62635366badfdfdcd1e29dfeacfda" exitCode=0
Jul 12 00:17:01.058386 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:01.058058    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g" event={"ID":"1547a8e7-59c0-4ad6-be99-f595b4c3d1cc","Type":"ContainerDied","Data":"290245adf244f682bcdf02ee1a51b9fdcbc62635366badfdfdcd1e29dfeacfda"}
Jul 12 00:17:02.193298 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:02.193264    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g"
Jul 12 00:17:02.282451 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:02.282389    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1547a8e7-59c0-4ad6-be99-f595b4c3d1cc-maas-api-service-ca\") pod \"1547a8e7-59c0-4ad6-be99-f595b4c3d1cc\" (UID: \"1547a8e7-59c0-4ad6-be99-f595b4c3d1cc\") "
Jul 12 00:17:02.282802 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:02.282781    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1547a8e7-59c0-4ad6-be99-f595b4c3d1cc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1547a8e7-59c0-4ad6-be99-f595b4c3d1cc" (UID: "1547a8e7-59c0-4ad6-be99-f595b4c3d1cc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:17:02.383714 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:02.383627    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1547a8e7-59c0-4ad6-be99-f595b4c3d1cc-maas-api-service-ca\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:03.070130 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:03.070097    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g"
Jul 12 00:17:03.070307 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:03.070097    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g" event={"ID":"1547a8e7-59c0-4ad6-be99-f595b4c3d1cc","Type":"ContainerDied","Data":"343a87d1c42bf4278d792fbe81c3027eec03869fefe20da395b206eac358d425"}
Jul 12 00:17:03.070307 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:03.070230    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="343a87d1c42bf4278d792fbe81c3027eec03869fefe20da395b206eac358d425"
Jul 12 00:17:03.964381 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:03.964350    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:17:03.999281 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:03.999195    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-envoy\") pod \"812118ce-0aed-4a5f-a3f4-681daeb32e74\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") "
Jul 12 00:17:03.999281 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:03.999237    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbsrb\" (UniqueName: \"kubernetes.io/projected/812118ce-0aed-4a5f-a3f4-681daeb32e74-kube-api-access-mbsrb\") pod \"812118ce-0aed-4a5f-a3f4-681daeb32e74\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") "
Jul 12 00:17:03.999281 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:03.999260    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-workload-socket\") pod \"812118ce-0aed-4a5f-a3f4-681daeb32e74\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") "
Jul 12 00:17:03.999561 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:03.999380    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/812118ce-0aed-4a5f-a3f4-681daeb32e74-istiod-ca-cert\") pod \"812118ce-0aed-4a5f-a3f4-681daeb32e74\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") "
Jul 12 00:17:03.999561 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:03.999441    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-podinfo\") pod \"812118ce-0aed-4a5f-a3f4-681daeb32e74\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") "
Jul 12 00:17:03.999561 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:03.999468    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-data\") pod \"812118ce-0aed-4a5f-a3f4-681daeb32e74\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") "
Jul 12 00:17:03.999561 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:03.999495    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-workload-certs\") pod \"812118ce-0aed-4a5f-a3f4-681daeb32e74\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") "
Jul 12 00:17:03.999561 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:03.999549    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-credential-socket\") pod \"812118ce-0aed-4a5f-a3f4-681daeb32e74\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") "
Jul 12 00:17:03.999561 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:03.999553    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "812118ce-0aed-4a5f-a3f4-681daeb32e74" (UID: "812118ce-0aed-4a5f-a3f4-681daeb32e74"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:17:03.999868 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:03.999597    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-token\") pod \"812118ce-0aed-4a5f-a3f4-681daeb32e74\" (UID: \"812118ce-0aed-4a5f-a3f4-681daeb32e74\") "
Jul 12 00:17:03.999868 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:03.999849    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/812118ce-0aed-4a5f-a3f4-681daeb32e74-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "812118ce-0aed-4a5f-a3f4-681daeb32e74" (UID: "812118ce-0aed-4a5f-a3f4-681daeb32e74"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:17:03.999972 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:03.999937    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "812118ce-0aed-4a5f-a3f4-681daeb32e74" (UID: "812118ce-0aed-4a5f-a3f4-681daeb32e74"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:17:04.000031 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:03.999966    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-workload-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:04.000031 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:03.999987    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/812118ce-0aed-4a5f-a3f4-681daeb32e74-istiod-ca-cert\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:04.000031 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.000005    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "812118ce-0aed-4a5f-a3f4-681daeb32e74" (UID: "812118ce-0aed-4a5f-a3f4-681daeb32e74"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:17:04.000387 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.000356    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-data" (OuterVolumeSpecName: "istio-data") pod "812118ce-0aed-4a5f-a3f4-681daeb32e74" (UID: "812118ce-0aed-4a5f-a3f4-681daeb32e74"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:17:04.001693 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.001672    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "812118ce-0aed-4a5f-a3f4-681daeb32e74" (UID: "812118ce-0aed-4a5f-a3f4-681daeb32e74"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:17:04.001904 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.001881    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/812118ce-0aed-4a5f-a3f4-681daeb32e74-kube-api-access-mbsrb" (OuterVolumeSpecName: "kube-api-access-mbsrb") pod "812118ce-0aed-4a5f-a3f4-681daeb32e74" (UID: "812118ce-0aed-4a5f-a3f4-681daeb32e74"). InnerVolumeSpecName "kube-api-access-mbsrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:17:04.002133 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.002097    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "812118ce-0aed-4a5f-a3f4-681daeb32e74" (UID: "812118ce-0aed-4a5f-a3f4-681daeb32e74"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 12 00:17:04.002133 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.002119    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-token" (OuterVolumeSpecName: "istio-token") pod "812118ce-0aed-4a5f-a3f4-681daeb32e74" (UID: "812118ce-0aed-4a5f-a3f4-681daeb32e74"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:17:04.079742 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.079704    2569 generic.go:356] "Generic (PLEG): container finished" podID="812118ce-0aed-4a5f-a3f4-681daeb32e74" containerID="5517e0ab17d07334a18b40e3778d8ba7111b2843fc8cf226ef0b795b09a7b7c1" exitCode=0
Jul 12 00:17:04.079902 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.079757    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"
Jul 12 00:17:04.079902 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.079790    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd" event={"ID":"812118ce-0aed-4a5f-a3f4-681daeb32e74","Type":"ContainerDied","Data":"5517e0ab17d07334a18b40e3778d8ba7111b2843fc8cf226ef0b795b09a7b7c1"}
Jul 12 00:17:04.079902 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.079847    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd" event={"ID":"812118ce-0aed-4a5f-a3f4-681daeb32e74","Type":"ContainerDied","Data":"27f361d693676855e8c8721b27c619ce09db0b0f13586fa35085ab0555a376ba"}
Jul 12 00:17:04.079902 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.079869    2569 scope.go:117] "RemoveContainer" containerID="5517e0ab17d07334a18b40e3778d8ba7111b2843fc8cf226ef0b795b09a7b7c1"
Jul 12 00:17:04.089981 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.089959    2569 scope.go:117] "RemoveContainer" containerID="af3d1a4cd36df93cd66f667b6d479d41caff5276ba1ffc4893d8bf05ad7ab484"
Jul 12 00:17:04.098145 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.098125    2569 scope.go:117] "RemoveContainer" containerID="5517e0ab17d07334a18b40e3778d8ba7111b2843fc8cf226ef0b795b09a7b7c1"
Jul 12 00:17:04.098405 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:17:04.098383    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5517e0ab17d07334a18b40e3778d8ba7111b2843fc8cf226ef0b795b09a7b7c1\": container with ID starting with 5517e0ab17d07334a18b40e3778d8ba7111b2843fc8cf226ef0b795b09a7b7c1 not found: ID does not exist" containerID="5517e0ab17d07334a18b40e3778d8ba7111b2843fc8cf226ef0b795b09a7b7c1"
Jul 12 00:17:04.098529 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.098417    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5517e0ab17d07334a18b40e3778d8ba7111b2843fc8cf226ef0b795b09a7b7c1"} err="failed to get container status \"5517e0ab17d07334a18b40e3778d8ba7111b2843fc8cf226ef0b795b09a7b7c1\": rpc error: code = NotFound desc = could not find container \"5517e0ab17d07334a18b40e3778d8ba7111b2843fc8cf226ef0b795b09a7b7c1\": container with ID starting with 5517e0ab17d07334a18b40e3778d8ba7111b2843fc8cf226ef0b795b09a7b7c1 not found: ID does not exist"
Jul 12 00:17:04.098529 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.098454    2569 scope.go:117] "RemoveContainer" containerID="af3d1a4cd36df93cd66f667b6d479d41caff5276ba1ffc4893d8bf05ad7ab484"
Jul 12 00:17:04.098771 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:17:04.098753    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af3d1a4cd36df93cd66f667b6d479d41caff5276ba1ffc4893d8bf05ad7ab484\": container with ID starting with af3d1a4cd36df93cd66f667b6d479d41caff5276ba1ffc4893d8bf05ad7ab484 not found: ID does not exist" containerID="af3d1a4cd36df93cd66f667b6d479d41caff5276ba1ffc4893d8bf05ad7ab484"
Jul 12 00:17:04.098866 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.098787    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af3d1a4cd36df93cd66f667b6d479d41caff5276ba1ffc4893d8bf05ad7ab484"} err="failed to get container status \"af3d1a4cd36df93cd66f667b6d479d41caff5276ba1ffc4893d8bf05ad7ab484\": rpc error: code = NotFound desc = could not find container \"af3d1a4cd36df93cd66f667b6d479d41caff5276ba1ffc4893d8bf05ad7ab484\": container with ID starting with af3d1a4cd36df93cd66f667b6d479d41caff5276ba1ffc4893d8bf05ad7ab484 not found: ID does not exist"
Jul 12 00:17:04.100588 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.100568    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-podinfo\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:04.100588 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.100588    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-data\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:04.100727 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.100597    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-workload-certs\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:04.100727 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.100606    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-credential-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:04.100727 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.100614    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-token\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:04.100727 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.100624    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/812118ce-0aed-4a5f-a3f4-681daeb32e74-istio-envoy\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:04.100727 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.100632    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbsrb\" (UniqueName: \"kubernetes.io/projected/812118ce-0aed-4a5f-a3f4-681daeb32e74-kube-api-access-mbsrb\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:04.108531 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.108503    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"]
Jul 12 00:17:04.108970 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.108948    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a7d1ef-openshift-default-5c9b7c8f9f-drfxd"]
Jul 12 00:17:04.983391 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:04.983354    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="812118ce-0aed-4a5f-a3f4-681daeb32e74" path="/var/lib/kubelet/pods/812118ce-0aed-4a5f-a3f4-681daeb32e74/volumes"
Jul 12 00:17:09.454606 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:09.454571    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2ea47-29730255-pmn95"]
Jul 12 00:17:09.456954 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:09.456925    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-a2ea47-29730255-pmn95"]
Jul 12 00:17:10.983067 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:10.983037    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bdbcfe0-684a-4190-94ef-02308e761382" path="/var/lib/kubelet/pods/9bdbcfe0-684a-4190-94ef-02308e761382/volumes"
Jul 12 00:17:24.827421 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:24.827386    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"]
Jul 12 00:17:24.827846 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:24.827685    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk" podUID="274302f9-dac6-4ac0-9e22-0c12651ee624" containerName="istio-proxy" containerID="cri-o://7ae0dd036fdadf6e14b219dbef45ab32b16e90cda1982c21201eb93c80638312" gracePeriod=30
Jul 12 00:17:30.081640 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.081612    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:17:30.217839 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.217753    2569 generic.go:356] "Generic (PLEG): container finished" podID="274302f9-dac6-4ac0-9e22-0c12651ee624" containerID="7ae0dd036fdadf6e14b219dbef45ab32b16e90cda1982c21201eb93c80638312" exitCode=0
Jul 12 00:17:30.217839 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.217799    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"
Jul 12 00:17:30.218032 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.217841    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk" event={"ID":"274302f9-dac6-4ac0-9e22-0c12651ee624","Type":"ContainerDied","Data":"7ae0dd036fdadf6e14b219dbef45ab32b16e90cda1982c21201eb93c80638312"}
Jul 12 00:17:30.218032 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.217933    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk" event={"ID":"274302f9-dac6-4ac0-9e22-0c12651ee624","Type":"ContainerDied","Data":"b14be02b1035133c8f8ff19610f9f98effca91c818ba00eba2ecc00991f965b1"}
Jul 12 00:17:30.218032 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.217963    2569 scope.go:117] "RemoveContainer" containerID="7ae0dd036fdadf6e14b219dbef45ab32b16e90cda1982c21201eb93c80638312"
Jul 12 00:17:30.227135 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.227123    2569 scope.go:117] "RemoveContainer" containerID="3f5c440b315105854f2286f2382448bcf684135d4c131a08633b2c64cffaa4e9"
Jul 12 00:17:30.234931 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.234913    2569 scope.go:117] "RemoveContainer" containerID="7ae0dd036fdadf6e14b219dbef45ab32b16e90cda1982c21201eb93c80638312"
Jul 12 00:17:30.235198 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:17:30.235181    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ae0dd036fdadf6e14b219dbef45ab32b16e90cda1982c21201eb93c80638312\": container with ID starting with 7ae0dd036fdadf6e14b219dbef45ab32b16e90cda1982c21201eb93c80638312 not found: ID does not exist" containerID="7ae0dd036fdadf6e14b219dbef45ab32b16e90cda1982c21201eb93c80638312"
Jul 12 00:17:30.235239 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.235211    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ae0dd036fdadf6e14b219dbef45ab32b16e90cda1982c21201eb93c80638312"} err="failed to get container status \"7ae0dd036fdadf6e14b219dbef45ab32b16e90cda1982c21201eb93c80638312\": rpc error: code = NotFound desc = could not find container \"7ae0dd036fdadf6e14b219dbef45ab32b16e90cda1982c21201eb93c80638312\": container with ID starting with 7ae0dd036fdadf6e14b219dbef45ab32b16e90cda1982c21201eb93c80638312 not found: ID does not exist"
Jul 12 00:17:30.235239 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.235229    2569 scope.go:117] "RemoveContainer" containerID="3f5c440b315105854f2286f2382448bcf684135d4c131a08633b2c64cffaa4e9"
Jul 12 00:17:30.235491 ip-10-0-136-83 kubenswrapper[2569]: E0712 00:17:30.235473    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f5c440b315105854f2286f2382448bcf684135d4c131a08633b2c64cffaa4e9\": container with ID starting with 3f5c440b315105854f2286f2382448bcf684135d4c131a08633b2c64cffaa4e9 not found: ID does not exist" containerID="3f5c440b315105854f2286f2382448bcf684135d4c131a08633b2c64cffaa4e9"
Jul 12 00:17:30.235547 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.235498    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f5c440b315105854f2286f2382448bcf684135d4c131a08633b2c64cffaa4e9"} err="failed to get container status \"3f5c440b315105854f2286f2382448bcf684135d4c131a08633b2c64cffaa4e9\": rpc error: code = NotFound desc = could not find container \"3f5c440b315105854f2286f2382448bcf684135d4c131a08633b2c64cffaa4e9\": container with ID starting with 3f5c440b315105854f2286f2382448bcf684135d4c131a08633b2c64cffaa4e9 not found: ID does not exist"
Jul 12 00:17:30.239772 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.239754    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-data\") pod \"274302f9-dac6-4ac0-9e22-0c12651ee624\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") "
Jul 12 00:17:30.239834 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.239788    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-workload-certs\") pod \"274302f9-dac6-4ac0-9e22-0c12651ee624\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") "
Jul 12 00:17:30.239834 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.239828    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-podinfo\") pod \"274302f9-dac6-4ac0-9e22-0c12651ee624\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") "
Jul 12 00:17:30.239922 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.239845    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/274302f9-dac6-4ac0-9e22-0c12651ee624-istiod-ca-cert\") pod \"274302f9-dac6-4ac0-9e22-0c12651ee624\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") "
Jul 12 00:17:30.239922 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.239877    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-token\") pod \"274302f9-dac6-4ac0-9e22-0c12651ee624\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") "
Jul 12 00:17:30.239922 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.239899    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-credential-socket\") pod \"274302f9-dac6-4ac0-9e22-0c12651ee624\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") "
Jul 12 00:17:30.240066 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.240000    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-workload-socket\") pod \"274302f9-dac6-4ac0-9e22-0c12651ee624\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") "
Jul 12 00:17:30.240066 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.240047    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "274302f9-dac6-4ac0-9e22-0c12651ee624" (UID: "274302f9-dac6-4ac0-9e22-0c12651ee624"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:17:30.240172 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.240109    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t85z2\" (UniqueName: \"kubernetes.io/projected/274302f9-dac6-4ac0-9e22-0c12651ee624-kube-api-access-t85z2\") pod \"274302f9-dac6-4ac0-9e22-0c12651ee624\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") "
Jul 12 00:17:30.240172 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.240141    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-envoy\") pod \"274302f9-dac6-4ac0-9e22-0c12651ee624\" (UID: \"274302f9-dac6-4ac0-9e22-0c12651ee624\") "
Jul 12 00:17:30.240172 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.240154    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "274302f9-dac6-4ac0-9e22-0c12651ee624" (UID: "274302f9-dac6-4ac0-9e22-0c12651ee624"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:17:30.240342 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.240315    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "274302f9-dac6-4ac0-9e22-0c12651ee624" (UID: "274302f9-dac6-4ac0-9e22-0c12651ee624"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:17:30.240342 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.240320    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/274302f9-dac6-4ac0-9e22-0c12651ee624-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "274302f9-dac6-4ac0-9e22-0c12651ee624" (UID: "274302f9-dac6-4ac0-9e22-0c12651ee624"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 12 00:17:30.240492 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.240454    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-workload-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:30.240492 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.240478    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-workload-certs\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:30.240607 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.240493    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/274302f9-dac6-4ac0-9e22-0c12651ee624-istiod-ca-cert\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:30.240607 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.240506    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-credential-socket\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:30.240711 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.240637    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-data" (OuterVolumeSpecName: "istio-data") pod "274302f9-dac6-4ac0-9e22-0c12651ee624" (UID: "274302f9-dac6-4ac0-9e22-0c12651ee624"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:17:30.242100 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.242077    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "274302f9-dac6-4ac0-9e22-0c12651ee624" (UID: "274302f9-dac6-4ac0-9e22-0c12651ee624"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 12 00:17:30.242206 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.242147    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "274302f9-dac6-4ac0-9e22-0c12651ee624" (UID: "274302f9-dac6-4ac0-9e22-0c12651ee624"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 12 00:17:30.242482 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.242461    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/274302f9-dac6-4ac0-9e22-0c12651ee624-kube-api-access-t85z2" (OuterVolumeSpecName: "kube-api-access-t85z2") pod "274302f9-dac6-4ac0-9e22-0c12651ee624" (UID: "274302f9-dac6-4ac0-9e22-0c12651ee624"). InnerVolumeSpecName "kube-api-access-t85z2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:17:30.242574 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.242478    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-token" (OuterVolumeSpecName: "istio-token") pod "274302f9-dac6-4ac0-9e22-0c12651ee624" (UID: "274302f9-dac6-4ac0-9e22-0c12651ee624"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 12 00:17:30.341393 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.341346    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-podinfo\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:30.341393 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.341386    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-token\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:30.341393 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.341396    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t85z2\" (UniqueName: \"kubernetes.io/projected/274302f9-dac6-4ac0-9e22-0c12651ee624-kube-api-access-t85z2\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:30.341393 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.341407    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-envoy\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:30.341393 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.341415    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/274302f9-dac6-4ac0-9e22-0c12651ee624-istio-data\") on node \"ip-10-0-136-83.ec2.internal\" DevicePath \"\""
Jul 12 00:17:30.541326 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.541289    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"]
Jul 12 00:17:30.544581 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.544555    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-a2ea47-openshift-default-74d57b6495-wbbsk"]
Jul 12 00:17:30.691615 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.691583    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qmhwn_59861bfe-ef22-4152-b1c1-ea4f3eba7cf4/manager/0.log"
Jul 12 00:17:30.806988 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.806961    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-pt2vk_9bad0275-0b31-4253-a626-88f6c74b1e91/manager/0.log"
Jul 12 00:17:30.925219 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.925184    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6657cc646c-k25f5_11d98225-9974-407e-b717-b818b2d14f99/manager/0.log"
Jul 12 00:17:30.984526 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:30.984494    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="274302f9-dac6-4ac0-9e22-0c12651ee624" path="/var/lib/kubelet/pods/274302f9-dac6-4ac0-9e22-0c12651ee624/volumes"
Jul 12 00:17:31.524561 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:31.524536    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d55578dd8-5z8ck_e77d5c53-f8da-4726-b9da-4528dcc3db5f/copy-manifests/0.log"
Jul 12 00:17:31.530841 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:31.530818    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d55578dd8-5z8ck_e77d5c53-f8da-4726-b9da-4528dcc3db5f/manager/0.log"
Jul 12 00:17:31.878220 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:31.878193    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-858b984599-f4m7g_047de06e-cc71-43c0-b81a-6b56cc5d3196/maas-api/0.log"
Jul 12 00:17:32.013922 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:32.013890    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29730225-nn99c_ceb9e99e-7f24-4f94-8f3c-a17214f8f00d/cleanup/0.log"
Jul 12 00:17:32.149464 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:32.149371    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29730240-4fxqb_c2ae38eb-40d0-4ad4-b3cd-b59261d1e07d/cleanup/0.log"
Jul 12 00:17:32.264868 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:32.264842    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29730255-rzkk7_435e74ad-4eb4-472e-9eac-40311e97f3f2/cleanup/0.log"
Jul 12 00:17:32.377405 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:32.377370    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-eb1c09fd-9bnwv_43341577-1d7f-4a0e-b0dd-a6ef1f73cfef/revoke-keys/0.log"
Jul 12 00:17:32.489063 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:32.488957    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-f85f8301-mwmck_e24a802e-136d-4d7a-a230-e386c90bc5ec/revoke-keys/0.log"
Jul 12 00:17:32.716818 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:32.716783    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-a58284-9wtdl_ac2bf81d-1eec-4a10-bfa1-3088a64e15a0/revoke-keys/0.log"
Jul 12 00:17:32.831556 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:32.831528    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-04a79e9d-qfjwn_9df1b096-2cfa-4954-869a-c4f4a9174e31/revoke-keys/0.log"
Jul 12 00:17:32.941563 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:32.941535    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-7007a3-6lcff_e3649950-0cd2-4a55-8582-7435fb3bc456/revoke-keys/0.log"
Jul 12 00:17:33.058759 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:33.058730    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-ad0409-ntmcm_1381f12e-957e-4f32-9883-a97016a0f7e2/revoke-keys/0.log"
Jul 12 00:17:33.176368 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:33.176286    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-da8e2e5f-66bmn_e4f4c213-6042-4db7-af2f-b85a24a91446/revoke-keys/0.log"
Jul 12 00:17:33.295959 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:33.295930    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-14eebdc1-ldwdx_f892bc36-40e9-4162-a5f6-0cc306ac17c9/revoke-keys/0.log"
Jul 12 00:17:33.411075 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:33.411048    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-64f2979f-h5wbb_16eb3877-fe85-4c93-abf1-e78c64a59f2f/revoke-keys/0.log"
Jul 12 00:17:33.531833 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:33.531763    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-a7d1ef-6kkwr_433ebee3-1422-47f5-acd4-2b0943f4d526/revoke-keys/0.log"
Jul 12 00:17:33.640510 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:33.640484    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-a2ea47-j5j9g_1547a8e7-59c0-4ad6-be99-f595b4c3d1cc/revoke-keys/0.log"
Jul 12 00:17:35.329521 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:35.329493    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-ffzdw_69e4114c-5e34-4a82-a58b-20456e9c9ace/manager/0.log"
Jul 12 00:17:36.315699 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:36.315673    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4s85pq_8b346891-a01d-4b8e-8246-afa554b6568b/istio-proxy/0.log"
Jul 12 00:17:36.435802 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:36.435773    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vdl2s_9d244dd8-877c-4b19-9a57-254eaff8b596/discovery/0.log"
Jul 12 00:17:36.545108 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:36.545081    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bd58cf797-5gb9m_8e1f7f37-08f8-4664-8d07-10b594f04655/kube-auth-proxy/0.log"
Jul 12 00:17:36.788294 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:36.788216    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hz4g6_73e09177-7cd0-493f-841d-252842535a58/istio-proxy/1.log"
Jul 12 00:17:36.918171 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:36.918140    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-skksg_828cf44d-5351-4eda-a056-0726258dabd1/payload-pre-processing/0.log"
Jul 12 00:17:37.033717 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:37.033684    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-98m9p_8b44e59d-5f12-470d-8b4a-1e3183c2f8f1/payload-processing/0.log"
Jul 12 00:17:37.503334 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:37.503303    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-wclk5_3d3bc893-8670-47d0-aff2-2e8d6b92a68e/main/0.log"
Jul 12 00:17:37.614922 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:37.614883    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-6pl8x_f263dd7b-87c9-410b-b14e-e24a4867f10b/main/0.log"
Jul 12 00:17:45.055948 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:45.055917    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-chgrx_faee1e90-99c1-4255-adb0-1d56308dbfd2/global-pull-secret-syncer/0.log"
Jul 12 00:17:45.234092 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:45.234056    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wh9hv_4ae214fa-346c-48ec-b3f7-67071c699d39/konnectivity-agent/0.log"
Jul 12 00:17:45.251814 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:45.251788    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-83.ec2.internal_6c9fb79e71283f9e071707f8e0898820/haproxy/0.log"
Jul 12 00:17:50.261360 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:50.261333    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-ffzdw_69e4114c-5e34-4a82-a58b-20456e9c9ace/manager/0.log"
Jul 12 00:17:52.770300 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:52.770265    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zc8ks_f27e72a2-5a95-4e89-a0a9-48e2c37b4985/node-exporter/0.log"
Jul 12 00:17:52.803944 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:52.803912    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zc8ks_f27e72a2-5a95-4e89-a0a9-48e2c37b4985/kube-rbac-proxy/0.log"
Jul 12 00:17:52.831979 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:52.831951    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zc8ks_f27e72a2-5a95-4e89-a0a9-48e2c37b4985/init-textfile/0.log"
Jul 12 00:17:55.526455 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.526413    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"]
Jul 12 00:17:55.526877 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.526760    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1547a8e7-59c0-4ad6-be99-f595b4c3d1cc" containerName="revoke-keys"
Jul 12 00:17:55.526877 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.526772    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1547a8e7-59c0-4ad6-be99-f595b4c3d1cc" containerName="revoke-keys"
Jul 12 00:17:55.526877 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.526787    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="812118ce-0aed-4a5f-a3f4-681daeb32e74" containerName="istio-proxy"
Jul 12 00:17:55.526877 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.526793    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="812118ce-0aed-4a5f-a3f4-681daeb32e74" containerName="istio-proxy"
Jul 12 00:17:55.526877 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.526804    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="274302f9-dac6-4ac0-9e22-0c12651ee624" containerName="istio-proxy"
Jul 12 00:17:55.526877 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.526810    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="274302f9-dac6-4ac0-9e22-0c12651ee624" containerName="istio-proxy"
Jul 12 00:17:55.526877 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.526822    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="274302f9-dac6-4ac0-9e22-0c12651ee624" containerName="istio-proxy"
Jul 12 00:17:55.526877 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.526828    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="274302f9-dac6-4ac0-9e22-0c12651ee624" containerName="istio-proxy"
Jul 12 00:17:55.526877 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.526876    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1547a8e7-59c0-4ad6-be99-f595b4c3d1cc" containerName="revoke-keys"
Jul 12 00:17:55.526877 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.526884    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="812118ce-0aed-4a5f-a3f4-681daeb32e74" containerName="istio-proxy"
Jul 12 00:17:55.527207 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.526891    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="274302f9-dac6-4ac0-9e22-0c12651ee624" containerName="istio-proxy"
Jul 12 00:17:55.527207 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.526898    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="274302f9-dac6-4ac0-9e22-0c12651ee624" containerName="istio-proxy"
Jul 12 00:17:55.530001 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.529985    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:17:55.532284 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.532258    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-scjg4\"/\"kube-root-ca.crt\""
Jul 12 00:17:55.532406 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.532356    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-scjg4\"/\"openshift-service-ca.crt\""
Jul 12 00:17:55.532483 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.532467    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-scjg4\"/\"default-dockercfg-hhxcj\""
Jul 12 00:17:55.535973 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.535949    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"]
Jul 12 00:17:55.553782 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.553753    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/754006ec-a839-49ba-a2d5-09f5bb707eac-lib-modules\") pod \"perf-node-gather-daemonset-hvmsm\" (UID: \"754006ec-a839-49ba-a2d5-09f5bb707eac\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:17:55.553916 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.553784    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/754006ec-a839-49ba-a2d5-09f5bb707eac-podres\") pod \"perf-node-gather-daemonset-hvmsm\" (UID: \"754006ec-a839-49ba-a2d5-09f5bb707eac\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:17:55.553916 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.553807    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/754006ec-a839-49ba-a2d5-09f5bb707eac-proc\") pod \"perf-node-gather-daemonset-hvmsm\" (UID: \"754006ec-a839-49ba-a2d5-09f5bb707eac\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:17:55.553994 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.553913    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg7pw\" (UniqueName: \"kubernetes.io/projected/754006ec-a839-49ba-a2d5-09f5bb707eac-kube-api-access-qg7pw\") pod \"perf-node-gather-daemonset-hvmsm\" (UID: \"754006ec-a839-49ba-a2d5-09f5bb707eac\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:17:55.553994 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.553951    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/754006ec-a839-49ba-a2d5-09f5bb707eac-sys\") pod \"perf-node-gather-daemonset-hvmsm\" (UID: \"754006ec-a839-49ba-a2d5-09f5bb707eac\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:17:55.655072 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.655037    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/754006ec-a839-49ba-a2d5-09f5bb707eac-proc\") pod \"perf-node-gather-daemonset-hvmsm\" (UID: \"754006ec-a839-49ba-a2d5-09f5bb707eac\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:17:55.655239 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.655096    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg7pw\" (UniqueName: \"kubernetes.io/projected/754006ec-a839-49ba-a2d5-09f5bb707eac-kube-api-access-qg7pw\") pod \"perf-node-gather-daemonset-hvmsm\" (UID: \"754006ec-a839-49ba-a2d5-09f5bb707eac\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:17:55.655239 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.655119    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/754006ec-a839-49ba-a2d5-09f5bb707eac-sys\") pod \"perf-node-gather-daemonset-hvmsm\" (UID: \"754006ec-a839-49ba-a2d5-09f5bb707eac\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:17:55.655239 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.655157    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/754006ec-a839-49ba-a2d5-09f5bb707eac-lib-modules\") pod \"perf-node-gather-daemonset-hvmsm\" (UID: \"754006ec-a839-49ba-a2d5-09f5bb707eac\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:17:55.655239 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.655165    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/754006ec-a839-49ba-a2d5-09f5bb707eac-proc\") pod \"perf-node-gather-daemonset-hvmsm\" (UID: \"754006ec-a839-49ba-a2d5-09f5bb707eac\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:17:55.655239 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.655172    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/754006ec-a839-49ba-a2d5-09f5bb707eac-podres\") pod \"perf-node-gather-daemonset-hvmsm\" (UID: \"754006ec-a839-49ba-a2d5-09f5bb707eac\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:17:55.655454 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.655274    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/754006ec-a839-49ba-a2d5-09f5bb707eac-lib-modules\") pod \"perf-node-gather-daemonset-hvmsm\" (UID: \"754006ec-a839-49ba-a2d5-09f5bb707eac\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:17:55.655454 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.655280    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/754006ec-a839-49ba-a2d5-09f5bb707eac-podres\") pod \"perf-node-gather-daemonset-hvmsm\" (UID: \"754006ec-a839-49ba-a2d5-09f5bb707eac\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:17:55.655454 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.655270    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/754006ec-a839-49ba-a2d5-09f5bb707eac-sys\") pod \"perf-node-gather-daemonset-hvmsm\" (UID: \"754006ec-a839-49ba-a2d5-09f5bb707eac\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:17:55.662916 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.662893    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg7pw\" (UniqueName: \"kubernetes.io/projected/754006ec-a839-49ba-a2d5-09f5bb707eac-kube-api-access-qg7pw\") pod \"perf-node-gather-daemonset-hvmsm\" (UID: \"754006ec-a839-49ba-a2d5-09f5bb707eac\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:17:55.841590 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.841545    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:17:55.977205 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:55.977161    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"]
Jul 12 00:17:55.982491 ip-10-0-136-83 kubenswrapper[2569]: W0712 00:17:55.982462    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod754006ec_a839_49ba_a2d5_09f5bb707eac.slice/crio-96d36833b0b25a4b6ce80c310eba152d8ad74320977f7e1432e78455a2971f92 WatchSource:0}: Error finding container 96d36833b0b25a4b6ce80c310eba152d8ad74320977f7e1432e78455a2971f92: Status 404 returned error can't find the container with id 96d36833b0b25a4b6ce80c310eba152d8ad74320977f7e1432e78455a2971f92
Jul 12 00:17:56.356582 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:56.356546    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm" event={"ID":"754006ec-a839-49ba-a2d5-09f5bb707eac","Type":"ContainerStarted","Data":"249983e6cc295bab699b056f6bb9b04c7e692c196146f4d27baf6b79c54779bc"}
Jul 12 00:17:56.356787 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:56.356609    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm" event={"ID":"754006ec-a839-49ba-a2d5-09f5bb707eac","Type":"ContainerStarted","Data":"96d36833b0b25a4b6ce80c310eba152d8ad74320977f7e1432e78455a2971f92"}
Jul 12 00:17:56.356787 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:56.356638    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:17:56.372021 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:56.371972    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm" podStartSLOduration=1.371956566 podStartE2EDuration="1.371956566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 00:17:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:17:56.371112139 +0000 UTC m=+2805.949204920" watchObservedRunningTime="2026-07-12 00:17:56.371956566 +0000 UTC m=+2805.950049341"
Jul 12 00:17:57.551635 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:57.551608    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jvtd6_b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf/dns/0.log"
Jul 12 00:17:57.571583 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:57.571563    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jvtd6_b3ea3f00-d48e-4ea8-8a9d-50c49ff4c9bf/kube-rbac-proxy/0.log"
Jul 12 00:17:57.690383 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:57.690357    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-smg2m_fc9214a0-6ad1-4315-a8d4-27b2f0a8d3a7/dns-node-resolver/0.log"
Jul 12 00:17:58.243860 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:58.243833    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7b7c4b75dd-lgsth_ee947d7e-27bc-4c75-b148-a2f95295c313/registry/0.log"
Jul 12 00:17:58.312843 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:58.312814    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mlkk5_ffcdbb3a-aa6c-4986-9a7f-fb26d0c8a790/node-ca/0.log"
Jul 12 00:17:59.249566 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:59.249538    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4s85pq_8b346891-a01d-4b8e-8246-afa554b6568b/istio-proxy/0.log"
Jul 12 00:17:59.359564 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:59.359531    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vdl2s_9d244dd8-877c-4b19-9a57-254eaff8b596/discovery/0.log"
Jul 12 00:17:59.378732 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:59.378708    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bd58cf797-5gb9m_8e1f7f37-08f8-4664-8d07-10b594f04655/kube-auth-proxy/0.log"
Jul 12 00:17:59.429615 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:59.429579    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hz4g6_73e09177-7cd0-493f-841d-252842535a58/istio-proxy/0.log"
Jul 12 00:17:59.458741 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:59.458718    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hz4g6_73e09177-7cd0-493f-841d-252842535a58/istio-proxy/1.log"
Jul 12 00:17:59.487244 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:59.487222    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-skksg_828cf44d-5351-4eda-a056-0726258dabd1/payload-pre-processing/0.log"
Jul 12 00:17:59.514587 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:17:59.514488    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-98m9p_8b44e59d-5f12-470d-8b4a-1e3183c2f8f1/payload-processing/0.log"
Jul 12 00:18:00.237049 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:00.237021    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-25dj7_03eb4687-6b09-4c00-8601-4f7e11f8de4c/serve-healthcheck-canary/0.log"
Jul 12 00:18:00.815662 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:00.815637    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ftcz2_82d38c08-74a5-42db-b8bb-c45a7b78417e/kube-rbac-proxy/0.log"
Jul 12 00:18:00.836668 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:00.836646    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ftcz2_82d38c08-74a5-42db-b8bb-c45a7b78417e/exporter/0.log"
Jul 12 00:18:00.859763 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:00.859742    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ftcz2_82d38c08-74a5-42db-b8bb-c45a7b78417e/extractor/0.log"
Jul 12 00:18:02.372632 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:02.372606    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-hvmsm"
Jul 12 00:18:02.978104 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:02.978071    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qmhwn_59861bfe-ef22-4152-b1c1-ea4f3eba7cf4/manager/0.log"
Jul 12 00:18:03.114867 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:03.114836    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-pt2vk_9bad0275-0b31-4253-a626-88f6c74b1e91/manager/0.log"
Jul 12 00:18:03.219736 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:03.219694    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6657cc646c-k25f5_11d98225-9974-407e-b717-b818b2d14f99/manager/0.log"
Jul 12 00:18:03.469774 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:03.469731    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d55578dd8-5z8ck_e77d5c53-f8da-4726-b9da-4528dcc3db5f/manager/0.log"
Jul 12 00:18:03.489662 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:03.489637    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d55578dd8-5z8ck_e77d5c53-f8da-4726-b9da-4528dcc3db5f/copy-manifests/0.log"
Jul 12 00:18:05.710272 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:05.710242    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4s85pq_8b346891-a01d-4b8e-8246-afa554b6568b/istio-proxy/0.log"
Jul 12 00:18:05.813941 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:05.813905    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vdl2s_9d244dd8-877c-4b19-9a57-254eaff8b596/discovery/0.log"
Jul 12 00:18:05.823748 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:05.823727    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bd58cf797-5gb9m_8e1f7f37-08f8-4664-8d07-10b594f04655/kube-auth-proxy/0.log"
Jul 12 00:18:05.865089 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:05.865061    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hz4g6_73e09177-7cd0-493f-841d-252842535a58/istio-proxy/0.log"
Jul 12 00:18:05.893055 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:05.893031    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hz4g6_73e09177-7cd0-493f-841d-252842535a58/istio-proxy/1.log"
Jul 12 00:18:05.911111 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:05.911088    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-skksg_828cf44d-5351-4eda-a056-0726258dabd1/payload-pre-processing/0.log"
Jul 12 00:18:05.929230 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:05.929162    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-98m9p_8b44e59d-5f12-470d-8b4a-1e3183c2f8f1/payload-processing/0.log"
Jul 12 00:18:07.148872 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:07.148785    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4s85pq_8b346891-a01d-4b8e-8246-afa554b6568b/istio-proxy/0.log"
Jul 12 00:18:07.267635 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:07.267604    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-vdl2s_9d244dd8-877c-4b19-9a57-254eaff8b596/discovery/0.log"
Jul 12 00:18:07.279897 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:07.279867    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bd58cf797-5gb9m_8e1f7f37-08f8-4664-8d07-10b594f04655/kube-auth-proxy/0.log"
Jul 12 00:18:07.315318 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:07.315288    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hz4g6_73e09177-7cd0-493f-841d-252842535a58/istio-proxy/0.log"
Jul 12 00:18:07.344983 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:07.344957    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-hz4g6_73e09177-7cd0-493f-841d-252842535a58/istio-proxy/1.log"
Jul 12 00:18:07.369380 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:07.369350    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-skksg_828cf44d-5351-4eda-a056-0726258dabd1/payload-pre-processing/0.log"
Jul 12 00:18:07.392987 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:07.392960    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-98m9p_8b44e59d-5f12-470d-8b4a-1e3183c2f8f1/payload-processing/0.log"
Jul 12 00:18:09.935514 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:09.935471    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-6fqjd_7cd949dd-1c5d-4fb8-83c6-82adff61ed58/migrator/0.log"
Jul 12 00:18:09.954965 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:09.954928    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-6fqjd_7cd949dd-1c5d-4fb8-83c6-82adff61ed58/graceful-termination/0.log"
Jul 12 00:18:11.397840 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:11.397812    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jwbv4_0835749c-5f54-425f-99de-161c4b66c81a/kube-multus-additional-cni-plugins/0.log"
Jul 12 00:18:11.418800 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:11.418764    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jwbv4_0835749c-5f54-425f-99de-161c4b66c81a/egress-router-binary-copy/0.log"
Jul 12 00:18:11.439114 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:11.439084    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jwbv4_0835749c-5f54-425f-99de-161c4b66c81a/cni-plugins/0.log"
Jul 12 00:18:11.458779 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:11.458756    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jwbv4_0835749c-5f54-425f-99de-161c4b66c81a/bond-cni-plugin/0.log"
Jul 12 00:18:11.478718 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:11.478686    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jwbv4_0835749c-5f54-425f-99de-161c4b66c81a/routeoverride-cni/0.log"
Jul 12 00:18:11.499138 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:11.499113    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jwbv4_0835749c-5f54-425f-99de-161c4b66c81a/whereabouts-cni-bincopy/0.log"
Jul 12 00:18:11.519673 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:11.519644    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jwbv4_0835749c-5f54-425f-99de-161c4b66c81a/whereabouts-cni/0.log"
Jul 12 00:18:11.725587 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:11.725502    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-nrn9c_43743170-d713-4977-a555-d59d7ab61f0c/kube-multus/0.log"
Jul 12 00:18:11.818613 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:11.818586    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5qc5j_7b557007-8a5a-47eb-a23e-a96ed97fb64c/network-metrics-daemon/0.log"
Jul 12 00:18:11.840294 ip-10-0-136-83 kubenswrapper[2569]: I0712 00:18:11.840267    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5qc5j_7b557007-8a5a-47eb-a23e-a96ed97fb64c/kube-rbac-proxy/0.log"